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Overview 


Thi s report summarizes the findings of a long-term evaluation of Opportunity NYC-Family Re- 
wards, an experimental, privately funded, conditional cash transfer (CCT) program to help families 
break the cycle of poverty. Family Rewards was the first comprehensive CCT program in a devel- 
oped country. Launched in 2007 by New York City’s Center for Economic Opportunity, it offered 
cash assistance to low-income families to reduce immediate hardship, but conditioned that assis- 
tance on families’ efforts to build up their “human capital” to reduce the risk of longer-term and 
second-generation poverty. The program thus tied a broad array of cash rewards (financial incen- 
tives) to prespecified activities and outcomes in the areas of children’s education, families’ preven- 
tive health care, and parents’ employment. It operated as a pilot program for three years, concluding, 
as planned, in August 2010. 

Six community-based organizations, in partnership with a lead nonprofit agency, ran Family Re- 
wards in six of New York City’s highest-poverty communities. MDRC evaluated the program 
through a randomized controlled trial involving approximately 4,800 families with 11,000 chil- 
dren; half of the families could receive the cash rewards if they met the required conditions, and half 
were assigned to a control group that did not participate in the program and could not receive the 
rewards. This report distills previously published findings and some longer-term updates on the pro- 
gram’s effects on a wide range of outcomes, covering two to six years after families entered the 
study (depending on the data source). 

Key Findings 

Family Rewards transferred over $8,700, on average, to families during the three-year period in 
which it operated. By the end of the study, it had produced some positive effects on some outcomes, 
but left many other outcomes unchanged. For example, the program: 

• Reduced current poverty and material hardship, including hunger and some housing-related 
hardships (especially for families in severe poverty), although those effects weakened after the cash 
transfers ended 

• Did not improve school outcomes for elementary or middle school students 

• Substantially increased graduation rates and other school outcomes for ninth-graders who en- 
tered high school as proficient readers, and increased their likelihood of subsequently enrolling 
full time in four-year colleges 

• Had few positive effects on school outcomes for nonproficient students 

• Did not increase families’ receipt of annual medical checkups, which was already high, and 
had limited effects on families’ health outcomes 

• Substantially increased families’ receipt of preventive dental care 

• Did not increase parents’ employment in or earnings from jobs covered by the unemployment 
insurance system (thus impeding sustained reductions in poverty), and led to some small earn- 
ings reductions for certain more disadvantaged subgroups 

MDRC is also studying a revised Family Rewards model that operated in Memphis, Tennessee, and 
the Bronx, New York, as part of a separate demonstration project, referred to as Family Rewards 2.0. 
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Preface 


In 2006, under fonner Mayor Michael R. Bloomberg, New York City officials began exploring 
new approaches to reduce poverty. One bold idea that captured their attention — conditional 
cash transfers (CCTs) — came from Mexico, and it was spreading throughout Latin America and 
other low- and middle-income regions of the world. The concept was fairly simple in principle: 
to reduce current poverty by making substantial cash payments to poor families now, while struc- 
turing those payments in a way that could reduce families’ longer-tenn poverty, especially the 
chances that their children would grow up poor. To receive the cash transfers, parents had to 
keep their children in school and get regular preventive health care for the whole family. 

This concept was transformational in Mexico, which had had no national cash transfer 
system previously. A rigorous evaluation showed that it reduced poverty and improved a num- 
ber of health and school outcomes for children. That evidence gave the program staying power 
across government administrations, and it propelled other countries to replicate the approach. 
Evaluation findings from multiple countries offered further support. 

Learning about this expanding international CCT movement, New York City officials 
wondered whether such a program could work in their city despite differences between its pov- 
erty and the largely rural poverty of Mexico, and the reality that New York City already had a 
well-established safety net (with important strands conditioned on work). To find out, the city’s 
Center for Economic Opportunity (CEO) commissioned MDRC to help it design and test a 
CCT approach, which came to be known as Opportunity NYC-Family Rewards. This three- 
year pilot program was launched in 2007 in six of the city’s highest-poverty communities. 
Broader in scope and more complex than other CCT programs, Family Rewards’ incentives 
were spread across three domains: children’s education, families’ preventive health care, and 
parents’ employment. It was bold, innovative, untested, and — given the broad-ranging opin- 
ions about fighting poverty with incentives — controversial. 

The results of a careful randomized trial are now in and paint a mixed picture. The pro- 
gram’s near-tenn reductions in poverty and, especially, in a range of material hardships, were its 
most impressive effects. And, overall, it achieved those effects without large unintended conse- 
quences, and despite transferring substantial amounts of money to participating low-income 
families. It also appeared to produce encouraging improvements in some academic and health- 
related outcomes for select participant subgroups. But it left most human capital outcomes — 
the primary targets of the program — unchanged. 

Still, some promising effects that emerged early in the study provided a foundation and 
rationale for trying again with a modified approach. In 201 1, MDRC and CEO redesigned and 
launched a test of a revised CCT model, dubbed “Family Rewards 2.0.” It operated for over 
three years (ending in 2014) in the Bronx, New York, and in Memphis, Tennessee. A report on 
its effects is scheduled for release in late 2016. 

During the design phase of the original demonstration, many choices about what activi- 
ties and accomplishments to reward, and how to structure and target the rewards, had to be 
made without the benefit of a strong evidence base. The lessons that are now available can help 
infonn further considerations of the use of incentives and CCT programs to help low-income 
families rise out of poverty, internationally as well as in the United States. 

Gordon L. Berlin 
President, MDRC 
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Executive Summary 


In 2006, during the administration of former Mayor Michael R. Bloomberg, New York City 
officials began to explore bold new ways of using financial incentives to address some of the 
root causes of poverty. City officials had become intrigued with the success of Mexico and a 
growing number of other lower- and middle-income countries in reducing poverty through 
incentives-based conditional cash transfer (CCT) programs, which provide cash rewards that are 
conditioned on participants’ achievement of specified behavioral benchmarks. They wondered 
whether such an approach could work in New York City. The newly established Center for 
Economic Opportunity (CEO), a unit within the Office of the Mayor, led the effort to find out. 

After an extensive consultation and design phase, supported in part by the Rockefeller 
Foundation and involving input from a wide range of city, national, and international experts 
(including MDRC and Seedco, a New York-based nonprofit workforce and economic devel- 
opment organization), CEO launched Opportunity NYC-Family Rewards, an experimental 
three-year CCT program for some of New York City’s most disadvantaged families. It selected 
Seedco to operate the program in partnership with six community-based organizations, and 
MDRC to evaluate it. A consortium of private funders paid for the cash transfers, the operation 
of the program, and the evaluation. 1 The program concluded, as planned, in 2010. 

This report recaps the history and design of Family Rewards and summarizes the main 
research findings. In doing so, it synthesizes MDRC’s previously published findings and pre- 
sents new results on selected outcomes over a longer follow-up period (many of which confirm 
early patterns). Depending on the data source, it covers program effects on parents and children 
for up to six years after the date when each family entered the study. A companion document 
provides more detail on the longer-term findings from the administrative data and is available as 
an online supplement to this report. 2 


The International CCT Movement 

Mexico pioneered the use of CCT programs. Its program, Prospera, was originally named Pro- 
gresa and then Oportunidades. It provides immediate relief to very poor families, mostly in rural 
areas, through direct cash payments that are conditioned in part on children remaining in school. 
These payments are intended as one effort to combat the problem of children leaving school 


'The funders of Opportunity NYC-Family Rewards are the Bloomberg Philanthropies, the Rockefeller 
Foundation, the Starr Foundation, the Open Society Institute, the Robin Flood Foundation, the Tiger Founda- 
tion, the Annie E. Casey Foundation, American International Group, and New York Community Trust. 

2 Cynthia Miller and Victoria Deitch, New York City’s First Conditional Cash Transfer Program: Wluit 
Worked, WhatDidn ’t — Supplemental Data on Impacts and Costs (New York: MDRC, 2016). 
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early, particularly in the middle grades, often to work in the fields to help their families earn 
much-needed income. It also requires families to get preventive health care and adopt certain 
child nutrition practices. In these ways, Prospera links basic income support with investments in 
families’ education, health, and employment potential — that is, their “human capital.” 

Today, CCT programs of varying scope and scale operate in most Latin American 
countries and in several countries in Africa, the Middle East, and Asia. Many of the programs 
serve as a core feature of a country’s social and economic “safety net,” and they typically condi- 
tion the cash transfers on children’s school attendance and families’ preventive health care prac- 
tices. Overall, studies have found CCT programs to improve school attendance, reduce short- 
term poverty, and increase consumption, particularly food consumption, with larger effects 
from programs that offered bigger transfers. 3 

A CCT Program for New York City: What Was Tried? 

New York City officials recognized that the urban poverty of its communities was vastly differ- 
ent from the largely rural poverty of Mexico and other low- or middle-income countries, and the 
reasons for children dropping out of school or performing poorly were not the same. Moreover, 
New York City, like the rest of the United States, already had a well-developed safety net, in- 
cluding income transfer programs for poor families. In contrast, Mexico had no national income 
support system, and its CCT program was instituted as the country’s main cash welfare system 
for poor families. A CCT program in New York City would thus have to be adapted to very 
different conditions. 

Following the basic tenet of all CCT programs, the intent of New York’s initiative was 
twofold: to use the rewards to reduce current family poverty and hardships, while simultaneous- 
ly supporting and encouraging families to invest in their own “human capital development” for 
their longer-term economic security. It tied cash rewards to a variety of activities and outcomes 
in three main domains: (1) children’s educational efforts and achievement, (2) families’ preven- 
tive health care practices, and (3) parents’ employment. The program operated in six of New 
York City’s highest-poverty communities, two each from the Bronx, Brooklyn, and Manhattan. 

The Family Rewards model was far more comprehensive and complex than CCT 
programs in other countries. It included 22 different financial rewards for various activities, 
which are summarized in Box ES. 1 . Values ranged from $20 per month for a parent maintain- 


3 Ariel Fiszbein and Norbert Schady, Conditional Cash Transfers: Reducing Present and Future Poverty 
(Washington, DC: The World Bank, 2009). 
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Box ES.1 

Types of Activities for Which Family Rewards Offered incentives 

Children’s Education 

Students: 

• High attendance (95 percent of scheduled days) 

• Proficient performance on standardized tests 

• Taking PSATs 

• Getting a library card 

• Earning required number of high school credits per year 

• Graduating from high school 

Parents: 

• Discussing test results with teachers 

• Attending parent-teacher conferences 

Family Preventive Health Care 

• Maintaining health insurance 

• Getting preventive medical and dental check-ups 

Parents’ Work and Training 

• Sustaining full-time work 

• Completing education/training 


ing public health insurance coverage for herself or himself to $600 for a high school student 
passing a New York State Regents (subject area) test. 4 

The program purposely included no case management. The goal was to test an “incen- 
tives-only” intervention. However, the program staff infonned families about other programs 
and services that they could try to access in the community that might help them achieve the 
outcomes for which Family Rewards offered cash incentives. Its staff also marketed the incen- 
tives heavily over the three-year program period in order to remind families of the CCT offer. 


4 Regents exams are administered to all public high school students in New York State. Students must pass 
at least five tests in specified subject areas in order to graduate with a diploma recognized by the New York 
State Board of Regents, which sets standards and regulations for all public schools. 
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The Evaluation 

Family Rewards was certainly not the first income transfer program in the United States to at- 
tach conditions to the receipt of benefits. The Earned Income Tax Credit (EITC), for example, 
conditions payments entirely on work. And the Temporary Assistance for Needy Families 
(TANF) program conditions cash assistance on parents’ efforts to look for work (unless they are 
exempted). 

Family Rewards established a much broader quid pro quo for benefits than these main- 
stream, work-based safety net programs. Its expansion of conditionality into the domains of ed- 
ucation and health, in addition to employment, with a large and diverse range of incentives, 
made it much more comprehensive and complex than any income transfer program ever operat- 
ed in the United States (and possibly anywhere). Neither its operational success nor its effects 
on families could be taken for granted. 

MDRC used a randomized controlled trial (RCT) to determine the program’s effects on 
families. The study involved approximately 4,800 families who applied for it, with 1 1,000 chil- 
dren (including elementary, middle, and high school students). Through a lottery-like process, 
about half of the applicant families were picked for Family Rewards and offered the incentives 
(the program group), and the balance were assigned to a control group that was not offered the 
incentives. As an RCT, the evaluation of Family Rewards measured the program’s effects, or 
“impacts,” by comparing the outcomes of the program and control groups; differences between 
the two groups’ outcomes that are statistically significant — that is, that are unlikely to have 
arisen by chance alone — can be attributed with a high degree of confidence to the program. 5 

Data for the evaluation came from administrative records and two waves of surveys of 
parents (at 1 8 months and at 42 months after random assignment), and a special survey of teen- 
agers at two years after random assigmnent. Other data, including findings from in-depth inter- 
views with families and staff, as well as program fiscal data, provided evidence on program op- 
erations and costs, families’ reward receipt patterns, and families’ understanding and views of 
the program. 

The evaluation found that Family Rewards transferred a substantial amount of cash to 
families, which reduced their current poverty and material hardship — its short-term goal. It 
also had a number of positive impacts on some participants’ human capital outcomes (in sup- 
port of its longer-term poverty-reduction goal), but it left most human capital outcomes un- 
changed for the majority of families. 


5 Unless otherwise noted, the impacts cited in this Executive Summary are statistically significant, with less 
than a 10 percent likelihood that they arose by chance alone. 
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Lessons Concerning Immediate Poverty Reduction 

• Family Rewards successfully implemented a broad-based quid pro quo 
in transferring substantial amounts of money to low-income families. 

The program passed a basic feasibility test: it transferred substantial amounts of money to 
low-income families, with all payments tied to a comprehensive and verifiable set of conditions. 
Virtually all families in the study’s program group earned and received rewards. On average, each 
family earned over $8,700 for all three years combined. Reward amounts averaged over $3,100 
during each of the first two years and $2,700 in the third year (when several rewards were discon- 
tinued). A majority of families — approximately 57 percent — earned at least $7,000 over the life 
of the program. The top 20 percent earned more than $13,000 in cash transfers. 

• Family Rewards reduced current poverty and material hardship, espe- 
cially for families in “severe poverty” — those with incomes that are less 
than half of the federal poverty level. 

Overall, when the program was operating, it succeeded in reducing current poverty. 
For example, counting the value of the reward payments, it boosted self-reported average 
household income for the program group by $353 per month in Year 3, an improvement of 
about 22 percent relative to the control group’s average monthly income of $1,620. This extra 
income reduced the proportion of families living at or below the federal poverty level by 12 
percentage points relative to the control group rate of 68 percent. 

The extra income helped those families reduce a variety of material hardships. For exam- 
ple, the proportion of families experiencing food insufficiency (meaning that parents indicated in 
survey interviews that their families “sometimes” or “often times” did not have enough to eat in 
the prior month) was 7 percentage points (or about 33 percent) lower than the control group aver- 
age of 22 percent, according to the 1 8-month survey (when all program group members were still 
enrolled in Family Rewards). The magnitude of this effect was somewhat smaller by the time of 
the 42 -month survey (about 5 percentage points). The parents in Family Rewards were also less 
likely (by 4 percentage points) than control group parents to report on the 42 -month survey that 
they did not have enough money to pay their rent at some point in the prior year. They were less 
likely (by 5.6 percentage points) to report at that time that they did not have enough money to 
“make ends meet.” Program group parents were about 18 percentage points more likely to report 
having a bank account after the program had ended. They were almost 8 percentage points more 
likely than control group parents to have any savings. And they were about 5 percentage points 
less likely to have borrowed cash from family or friends in the prior year. 

Hardship reductions tended to be more concentrated among families who were living in 
severe poverty when they entered Family Rewards. For example, according to the 42-month 
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survey, the program reduced the proportion of that subgroup reporting food insufficiency by 9 
percentage points (or by 40 percent) relative to similar families in the control group, and it re- 
duced the proportion that had not paid their full rent in the past year by 1 1 percentage points (or 
23 percent). In contrast, reductions in hardships were small and statistically insignificant among 
those whose poverty was not as severe at the time they began the program. 

The reductions in material hardship that are evident in the early post-program period (as 
indicated by the 42 -month survey results, roughly within the six months after the rewards offer 
ended) most likely reflected a lingering effect of the cash transfers. They were not indicative of 
a pennanent change. 

• The poverty and hardship reductions that Family Rewards had achieved 
began to attenuate after the cash transfers ended. 

Most of the poverty reduction observed during the program period was attributable to 
the cash transfers that families received. Once Family Rewards ended and the transfers were no 
longer available, the average income of families in the program group no longer differed sub- 
stantially from that of the control group. This outcome was a direct consequence of the failure 
of the program to lead to increased parental employment and earnings. 


Lessons Concerning Human Capital Development 

Although Family Rewards sought to reduce current poverty in a way that would improve the 
human capital of parents and children, to help them avoid future poverty, this goal proved hard- 
er to achieve. Some participants, especially teenagers who were already better students when 
they entered the program (that is, they had scored proficient or higher on their eighth-grade 
standardized tests), benefited from Family Rewards in a number of important ways. On most 
measures, however, and for the majority of families, the program had no positive effects or pro- 
duced only small improvements. 

A Few Positive Results 

• Family Rewards increased high school achievement and graduation 
rates for initially better-prepared students, and increased their likeli- 
hood of full-time enrollment in four-year colleges. 

Among students who were entering ninth grade when their families enrolled in Family 
Rewards, those who scored at or above proficiency levels on their eighth-grade standardized 
math and reading-focused English language arts (ELA) tests performed better in high school 
than they would have in the absence of the program. For example, these students achieved high- 
er attendance rates, accumulated more course credits, and passed more subject-area Regents 
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exams than similarly proficient students in the control group. Moreover, as a result of Family 
Rewards, of the ninth-graders who were deemed reading - proficient when they entered high 
school (who made up about 31 percent of the overall sample of ninth-graders), 82 percent grad- 
uated within six years, a rate that was nearly 10 percentage points higher than the rate for simi- 
lar students in the control group. 

A special study that was conducted as part of the evaluation revealed that high school 
students in Family Rewards shifted more of their time during nonschool hours from social ac- 
tivities to academic activities (for example, homework and achievement-oriented after-school 
activities), compared with the way students in the control group spent their time. This change 
was greater among the more proficient students. As such, it may be one of the mechanisms 
through which the program improved the school outcomes for such students. 

For the subgroup of ninth-graders who were initially proficient in reading, the positive 
schooling effects of Family Rewards carried over somewhat to postsecondary education. Al- 
though the program did not increase their overall enrollment in college and other postsecondary 
institutions, it changed the patterns of their engagement. For example, it increased full-time en- 
rollment m four-year institutions by 9 percentage points above the 41 percent rate observed for 
comparable students in the control group. 

• Family Rewards reduced self-reported problem behaviors among 
teenagers. 

The special study of teenagers’ time use also examined the effects of Family Rewards 
on teenagers’ mental health and problem behaviors. It found that teenagers in the program 
group reported lower levels of aggression and lower levels of alcohol and marijuana use, and a 
lower likelihood of having friends using those substances, compared with their control group 
peers. For example, they were about 10 percentage points less likely to report having engaged in 
behaviors that were intended to cause harm to others, and 15 percentage points less likely to 
report having engaged in any substance abuse in the prior month. 6 These effects were observed 
regardless of the teenagers’ educational proficiency levels. 

• Family Rewards increased families’ receipt of dental care 

Family Rewards led to increased dental care for parents and children alike. According 
to responses to the 42 -month survey, parents in the program group were 10 percentage points 
more likely than control group parents to report having seen a dentist for any reason in the prior 


6 Pamela Morris, J. Lawrence Aber, Sharon Wolf, and Juliette Berg, Using Incentives to Change How 
Teenagers Spend Their Time: The Effects of New York City’s Conditional Cash Transfer Program (New York: 
MDRC, 2012). 
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year, and about 12 percentage points more likely to have had two dental checkups or more in 
the past year. The program also had strong positive effects on dental care among high school 
students and younger children. 

Disappointing Results 

• Family Rewards had no positive effects on school outcomes for elemen- 
tary school students or for middle school students. 

For elementary and middle school students, the evaluation found no pattern of positive 
effects on attendance rates, scores on standardized tests, grade progression, or other school out- 
comes during the program period or after six years from the time of random assigmnent. In ad- 
dition, subgroup analyses did not reveal any consistent patterns of positive effects for particular 
types of students in those grades. Perhaps the program model’s limited approach for these chil- 
dren — primarily rewarding attendance and standardized test scores (rather than more immedi- 
ate performance indicators, such as good report card grades), and paying all rewards to the par- 
ents — might help explain why Family Rewards did not benefit students who were in 
elementary and middle school when they entered the program. 

• Family Rewards had no positive effects on school outcomes for less pro- 
ficient ninth-graders. 

Ninth-grade students who were behind educationally when they entered Family Rewards 
(that is, they had scored below proficiency levels on their eighth-grade reading or math tests) did 
not experience any educational gains from the program. It may be that the members of this 
group, who made up about two-thirds of the overall sample of ninth-graders, were just too poorly 
prepared for high school or too disengaged from school for the incentives to make a difference. 

• Family Rewards did not improve the receipt of preventive health care 
services for most fa mili es. 

The health-related incentives of the Family Rewards program were designed to encour- 
age low-income families to adopt better preventive health care practices. It turned out that a 
higher proportion of families than the program’s designers had expected were already receiving 
health insurance coverage and practicing preventive health care. 7 The high rate in New York 
City, which was higher than in many other cities, likely reflects prior efforts by the city and state 
to improve access to health care coverage and to improve the health care delivery system for 
low- and moderate-income families. Most parents in the control group reported having health 
insurance for their families, reported getting regular physicals for themselves and their children, 


7 The study sample did not include low-income single adults or undocumented immigrants, who are much 
less likely to have health insurance. 
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and said they had a regular place for receiving routine health services (a “medical home”). Con- 
sequently, on some measures, Family Rewards had much less room to improve the use of basic 
preventive health care services further. 

• Family Rewards did not lead parents to increase their employment or 
earnings in jobs covered by the unemployment insurance (UI) system. 

According to the 42-month survey of parents, the program increased the likelihood 
of working at the time of the interview by over 6 percentage points above the control group 
rate of 50 percent. This difference was driven largely by an increase in full-time work 
(which the program rewarded). However, according to administrative records data, the pro- 
gram had no statistically significant impact on the average quarterly rate of employment in 
Ul-covered jobs or on average earnings. (Some jobs, such as private child care, are not cov- 
ered by the UI system.) 

At the same time, subgroup analyses revealed that Family Rewards had a statistically 
significant negative effect on labor market outcomes for parents who entered the program with 
lower education levels and other disadvantages; in other words, they worked and earned less 
than they would have in the absence of the program, according to UI records. For example, par- 
ents who did not have a high school diploma or General Educational Development (GED) cer- 
tificate had an average quarterly employment rate over the five years after random assignment 
that was 3 percentage points lower than that of their counterparts in the control group, and they 
earned an average of $2,934 less (a reduction of almost 8 percent). 

The small reductions in Ul-covered work among more disadvantaged parents may be a 
response to the substantial rewards that these families were earning from the program’s health 
and education domains — an “income effect.” Since families were able to earn program re- 
wards by completing education or health-related activities, parents who were less prepared for 
the labor market may have relied more on earning what they could from those two program 
components and less on looking for or maintaining full-time work. 


Some Exploratory Findings 

As a comprehensive assessment of a new and complex intervention, the Family Rewards eval- 
uation includes a number of exploratory analyses of hypotheses that emerged while the study 
was under way. Although the results from these analyses do not carry the same weight as the 
study’s main findings, they suggest some other possible ways in which a CCT program might 
influence families, and that might be worth testing in future studies. Two of these analyses are 
highlighted here. 
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• Family Rewards may have improved school achievement of younger 
siblings. 

A sizable number of families in the Family Rewards evaluation had very young chil- 
dren, who ranged in age from preschoolers to kindergarteners when their parents enrolled in the 
study. These children were eligible for the health rewards but not the education rewards, unless 
they reached third grade during the three-year program period. However, they would have expe- 
rienced increased family incomes during their preschool or early elementary school years from 
the rewards received by their parents and older siblings. A question to consider is whether living 
in a family with less poverty at that younger age improves their subsequent educational out- 
comes. The current study cannot answer that question definitively, but it offers some suggestive 
evidence. 

By the fifth and sixth years after random assignment, many of these children were old 
enough to have taken standardized tests, which are administered starting in third grade. Accord- 
ing to an exploratory analysis, Family Rewards increased the proportion of these children who 
scored at proficiency levels on the ELA test in Year 6 by 5 percentage points, and on the math 
test by nearly 6 percentage points. Although the mechanisms for these impacts are not known, 
the pattern is consistent with findings from a number of “welfare-to-work” experiments that 
showed that program-produced increases in family income could have positive effects on young 
children’s school performance. 8 Given the exploratory nature of this analysis, additional evi- 
dence would be required to confirm these findings. 

• Family Rewards may have improved health outcomes for parents with 
more health problems than other sample members. 

It is reasonable to expect that family members who were healthy when they entered 
Family Rewards would not see much improvement in their health status during the study’s 
follow-up period, although the program might help them stay healthy through its emphasis on 
preventive care. In contrast, those who were less healthy to begin with would have more room to 
improve their health outcomes. An exploratory subgroup analysis found some indications of dif- 
ferent effects for these groups. For example, parents who at the time of random assignment rated 
themselves as being in “fair” or “poor” health (about 19 percent of the sample) were 6 percentage 
points more likely than similar parents in the control group (or almost twice as likely) to report 
that they were in “very good” or “excellent” health at the time of the 42-month survey. No such 
effect was observed for the healthier subgroup. However, parents in the healthier subgroup were 
more likely than similar parents in the control group (by 4 percentage points) to report being 
treated for a medical condition (perhaps one that had previously gone undiagnosed), such as 


8 Pamela A. Morris, Aletha C. Huston, Greg J. Duncan, Danielle A. Crosby, and Johannes M. Bos, How 
Welfare and Work Policies Affect Children: A Synthesis of Research (New York: MDRC, 2001). 
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asthma. 9 Although not large or definitive, these findings suggest that future studies of health- 
related incentives should take initial health status into account. 


What’s Next? 

Although it is impossible to know for sure, it is reasonable to question whether certain design 
choices help explain why Family Rewards did not produce larger and more widespread positive 
effects on human capital outcomes. For example, were the right behaviors and achievements 
rewarded? Were they structured and operationalized appropriately? Were there too many re- 
wards overall? Were appropriate dollar values attached to the rewards? Should case manage- 
ment have been included? Chapter 5 of this report explores these and other questions. 

But Opportunity NYC-Family Rewards was not the end of the efforts to experiment 
with a CCT approach in the United States. With the early evidence pointing to some promising 
effects of the original model, but also revealing a number of limitations, CEO and MDRC sought 
to try again in 2011 with a modified approach, even though the final results from the first attempt 
would not be available for several years. They collaborated to redesign Family Rewards and se- 
cured funding to test it with a new randomized trial. Support came from the federal Social Inno- 
vation Fund and private funds. 10 This time, the program would be tested in two very different 
locations: the Bronx, New York, and Memphis, Tennessee. It operated from 201 1 through 2014. 

The revised model, referred to as Family Rewards 2.0, is notable in a number of ways. 
While still a three-year program, it included many fewer incentives (8 rather than 22), with 
some dropped after it was determined that they were not working very well based on very early 
indications emerging from the original demonstration. The revised program also targeted only 
families with high school students in grades 9 or 10, since no positive effects on school out- 
comes had been found for elementary and middle school students. It further restricted the selec- 
tion of families to those who were receiving TANF or SNAP benefits, as a way to focus atten- 
tion on families who were among the poorest and for whom control group outcomes in the 
original study tended to be worse than for those who were not participating in these benefit pro- 
grams. This targeting decision was also seen as a way of testing a program that offered the po- 
tential of a more direct connection to two of the country’s mainstream safety net programs. 

The new model also included a “family guidance” component, whereby each family 
was assigned a family adviser. The decision to include this feature was based on the emerging 


9 The difference in impacts across subgroup categories was statistically significant for the measure “being 
treated for a medical condition” but not for the self-rated health measure. 

10 The Social Innovation Fund is a program of the Corporation for National and Community Service that 
combines public and private resources to replicate and further test promising innovations to improve the lives 
of people in low-income communities. 
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findings from the original study suggesting that incentives by themselves would not be suffi- 
cient to produce big gains in human capital development. 

Finally, Family Rewards 2.0 also attempted to make the reward payments more timely 
(and thus more salient) by disbursing payments monthly rather than every two months. Reward- 
ing good report card grades, in addition to test scores, provided a more immediate incentive for 
teenagers to fry to improve their school perfomiance, since these rewards were tied to course 
grades in each report card. 

The original Family Rewards experiment revealed a number of challenges in trying to 
use a two-generation, incentives-only strategy to achieve the dual goals of immediate poverty 
reduction and family human capital development. Whether the revised approach represented by 
Family Rewards 2.0, which builds on but also departs substantially from the initial model, is 
more effective remains to be seen. Impact findings on that project will be available in late 2016. 
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Chapter 1 

Introduction 


During the administration of fonner Mayor Michael R. Bloomberg, New York City officials 
sponsored a major new antipoverty initiative called Opportunity NYC- Family Rewards, a con- 
ditional cash transfer (CCT) program to help families break the cycle of intergenerational pov- 
erty. The first of its kind in the United States, Family Rewards tied cash rewards for very-low- 
income families to a variety of activities and outcomes related to children’s educational efforts 
and achievement, families’ preventive health care practices, and parents’ employment. The in- 
tent of this three-year demonstration project was twofold: (1) to use the rewards to reduce cur- 
rent family poverty and hardships, while (2) simultaneously supporting and encouraging fami- 
lies to invest in their own education, health, and employment potential — or “human capital 
development” — for their longer-term economic security. 

The New York City Center for Economic Opportunity (CEO) was assigned overall re- 
sponsibility for Family Rewards, 1 which was operated in six of New York’s highest-poverty 
communities by nonprofit organizations. MDRC conducted a comprehensive evaluation of the 
program. The study showed that, while the cash transfers were available, the program substan- 
tially reduced family poverty and material hardship, particularly among the poorest families. It 
also produced positive effects on some human capital outcomes, including higher graduation 
rates among some high school students and some improvements in certain health-related indica- 
tors. However, it also left most important outcomes unchanged. 

This report recaps the history and design of the program and summarizes the main re- 
search findings. In doing so, it synthesizes previously published findings and presents new re- 
sults on selected outcomes over a longer follow-up period (many of which confirm early pat- 
terns). Depending on the data source, it covers program effects on participants’ outcomes for up 
to six years after the date when each family entered the study. 2 A companion document pro- 
vides more detail on the longer-term findings from the administrative data and is available as an 
online supplement to this report. 3 


'CEO is a unit within the Office of the Mayor charged with building and applying evidence on innovative 
policies to reduce poverty and promote economic mobility among lower-income New Yorkers. 

2 These past studies are referenced throughout this document and are listed at the back of this report. See 
also Miller et al. (2015) for a summary of earlier impact findings. 

3 See Miller and Deitch (2016), available at www.mdrc.org. 
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Origins of the Family Rewards Demonstration 

In 2006, New York City officials began to explore bold new ways of using financial incentives 
to address some of the root causes of poverty. They initiated this effort after learning about suc- 
cessful efforts with CCT programs in Latin America and a growing number of lower- and mid- 
dle-income countries throughout the world. Mexico pioneered this movement, and its experi- 
ence was known best. It launched its first program, called Progresa, in 1997, and as it continued 
to evolve, renamed the program Oportunidades in 2002 after a change in government, and then 
Prospera in 2014 under a new government. The program provides immediate relief to very poor 
families, mostly in rural areas, through direct cash payments that are conditioned in part on 
children remaining in school. These payments are intended as one effort to combat the problem 
of children leaving school early, particularly in the middle school grades, often to work in the 
fields to help their families earn much-needed income. Prospera also offers payments to families 
who get preventive health care and adopt certain child nutrition practices. 4 In these ways, Pros- 
pera links basic income support with investments in families’ human capital. 

Today, CCT programs of varying scope and scale operate in most Latin American 
countries, as well as in several countries in Africa, the Middle East, and Asia. Most of the pro- 
grams, many of which serve as a core feature of a country’s economic and social safety net, of- 
fer incentives for education (for school-age children) and preventive health care, while a few 
target only education. Some are offered broadly to all low-income families in an area, while 
others are targeted to specific populations of concern. The programs overall have been found to 
reduce short-term poverty and increase consumption, particularly food consumption, with larger 
effects from programs that offer bigger transfers, and a mix of results for other outcomes. 5 

Adapting the CCT Concept to New York City 

Inspired by the examples of Mexico and other countries, New York City officials won- 
dered whether some form of a CCT program could work in their own city, and they began to 
explore the idea of a trial project. Of course, the urban poverty of New York City is vastly dif- 
ferent from the largely rural poverty of Mexico and other low-income countries, and the reasons 
for children dropping out of school or performing poorly are not the same. Moreover, New 
York City, like the rest of the United States, already has a well-developed social safety net, in- 


4 For a comprehensive review of the history and evaluation of Progresa, see Levy (2006). 

5 For a detailed review of CCT programs worldwide and a synthesis of available evaluation findings, see 
Fiszbein and Schady (2009). Summarizing those findings, Fiszbein and Schady state (pp. 3-4): “CCTs have led 
poor households to make more use of health and education services, a key objective for which they were de- 
signed. Nevertheless, the evidence on improvements in final outcomes in health and education is more mixed.” 
The authors point to the findings such as positive effects on school enrollment but not on learning outcomes as 
measured by achievement tests, and although some CCT programs have produced positive health outcomes, 
such as reduced stunting (of physical growth) and improved nutrition, others have not. 
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eluding income support systems for poor families. In contrast, Mexico had no national income 
support system, and its CCT program was instituted as the country’s main cash welfare system 
for poor families . 6 A CCT program in New York City would thus have to be adapted to very 
different conditions. Still, the basic principle of CCTs — structuring cash transfers in a way that 
promotes human capital development while simultaneously alleviating immediate poverty and 
hardship — was compelling. 

To explore the feasibility and potential value of a CCT program for New York City, 
CEO entered into a partnership with the Rockefeller Foundation, which envisioned that such a 
project could provide a new opportunity to help low-income New Yorkers while also building 
evidence on a poverty reduction strategy that would have national and international importance. 
With a special grant from the Rockefeller Foundation, CEO commissioned MDRC to help de- 
sign the project, including a rigorous evaluation. Subsequently, CEO and MDRC engaged 
Seedco, a nonprofit workforce and economic development organization, in the planning pro- 
cess, and the three groups worked closely together to come up with a design for a CCT demon- 
stration project. During this planning process, they conferred with officials and researchers in- 
volved with Mexico’s Oportunidades program, meeting with them in New York City and 
visiting their program in Mexico. The designers also sought guidance and feedback on the idea 
from the World Bank; the Inter-American Development Bank; experts in universities, founda- 
tions, other social policy organizations; and officials in various New York City government 
agencies. 

The design team also reviewed evidence on other types of incentives programs in the 
United States and in other high-income countries. These programs tend to differ from most 
CCT programs in less affluent countries in that they are usually targeted to one area or type of 
outcome, such as workforce outcomes, health outcomes, or school outcomes, and (with the ex- 
ception of some workforce wage supplementation programs) did not provide large cash trans- 
fers with the goal of reducing immediate poverty and hardship. They targeted and had some 
success in improving such outcomes as perfomiance on standardized tests, taking and passing 
Advanced Placement courses, taking a full-time college course load, losing weight, quitting 
smoking, working full time, or participating in a training program. 

The Distinctiveness of Family Rewards 

The Family Rewards model that emerged from this design process is one that was far 
more comprehensive than most incentive-based programs previously tried in the United States, 
and more comprehensive and complex than CCTs in other countries. The rewards were struc- 
tured into three main domains: 


6 Mexico’s program serves about five million households, accounting for almost a fifth of the entire Mexi- 
can population (Fiszbein and Schady, 2009). 
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• Education-focused conditions, which included meeting goals for children’s 
attendance in school, achievement levels on standardized tests, and other 
school progress markers, as well as parents’ engagement with their children’s 
education 

• Health-focused conditions, which included maintaining health insurance 
coverage for parents and their children, as well as obtaining age-appropriate 
preventive medical and dental checkups for each family member 

• Work-focused conditions, aimed at parents, which included sustaining full- 
time work and participation in approved education or job training activities 

An important feature of Family Rewards was that it purposely did not include case 
management. The goal was to test an “incentives-only” intervention. However, the program did 
make infonnation available to families on services and other programs that they could try to 
access in the community to help them achieve the outcomes that Family Rewards encouraged. 
Its staff also marketed the incentives heavily in order to remind families of the offer that was 
available to them over the three-year program period. 

City officials anticipated that a CCT program in New York City would be controversial 
(as discussed below) and that funding this unconventional approach with public dollars, even on 
a trial basis, would be widely opposed. Consequently, they sought to support the demonstration 
with private resources. 7 After initial funding was secured, CEO granted final approval for the 
project in the late spring of 2007. Agreements were established for Seedco to operate the pro- 
gram and for MDRC to evaluate it. 8 Most of the families who were recruited for the study were 
enrolled by the end of 2007, with early enrollees beginning to earn rewards in September of 
that year. 


7 Family Rewards was one of three incentives-based poverty reduction strategies launched by CEO in 2007 
under the rubric of Opportunity NYC. The other two are Work Rewards, a workforce program for recipients of 
government rent subsidies from New York City’s Housing Choice Voucher (Section 8) program, and Spark, a 
school-based education incentives program for fourth- and seventh-graders, which was evaluated by Harvard 
Education Labs. Funders of the Opportunity NYC demonstration include the Bloomberg Philanthropies, the 
Rockefeller Foundation, the Starr Foundation, the Open Society Institute, the Robin Hood Foundation, the 
Tiger Foundation, the Annie E. Casey Foundation, American International Group, the John D. and Catherine 
T. MacArthur Foundation (for Work Rewards only), New York Community Trust, and the Broad Foundation 
(for Spark only). 

8 For this project, MDRC was a contractor to the Mayor’s Fund for the City of New York, a 501(c)(3) entity 
through which private hinds can be donated to the city for specific charitable uses. CEO provided general direc- 
tion and oversight, in cooperation with the Mayor’s Fund. Seedco was a subcontractor to MDRC. 
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Controversies 

From the time of its announcement, the idea of a CCT program for New York City was contro- 
versial. Critics pointed to possible negative effects that could arise from this type of interven- 
tion, including ramifications that could extend beyond the program’s effects on the participating 
families. 

Some critics argued vociferously that “paying” people to be “good parents” and to “be- 
have responsibly” is morally wrong and may weaken norms that signal and guide socially ap- 
propriate behavior for parents and children. While believing that poor families’ own motiva- 
tional problems or bad choices contribute to their lower incomes, lower educational attainment, 
and, in some cases, health problems, such critics contended that offering incentives was not the 
right way to try to change that behavior. Some expressed a fear that, if offered on a larger scale, 
this approach would feed expectations among low-income families — and perhaps others — 
that they should be paid for fulfilling their “moral obligations” as parents and citizens . 9 

Some scholars strongly oppose attaching financial rewards to education in general, ar- 
guing that such an approach can harm children’s educational achievement over time. Drawing 
largely on psychological theory and laboratory-like experiments, these scholars contend that 
extrinsic rewards can undermine an intrinsic motivation to achieve particular goals — including 
the motivation to leam — and that persistence in pursuing those goals may weaken once the 
rewards are terminated. For example, the desire to leam for its own sake may be diluted when 
students are rewarded with money or prizes to engage in learning activities . 10 Some critics also 
fear that participants who have difficulty earning rewards may become discouraged and be 
more likely give up trying to achieve certain outcomes as a result. But other scholars have chal- 
lenged claims that external rewards undermine intrinsic motivation . 11 

Some commentators objected that Family Rewards would distract attention from what 
they deem to be the main causes of poverty, which are more structural in nature — for example, 
poor schools, poor services, and a lack of good jobs. They were thus skeptical that the program 
could make a substantial difference. Some also viewed it as paternalistic and “insulting” to poor 
parents to assume that they need to be paid to do what is right for their families . 12 

City officials had a different view. They wanted to try new strategies to tackle persis- 
tent, entrenched poverty where other approaches had failed, and that, given the positive results 
that other CCT programs and incentive-based strategies have sometimes achieved, they be- 
lieved that this approach was worth testing in New York City. The evidence would show, for 


9 See, for example, Rosenberg (2008). 

10 See, for example, Deci, Koestner, and Ryan (1999). 

1 'See, for example, Cameron and Pierce (2002). 

12 For an example of contrasting views, see Becker (2008). 
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example, whether students’ achievement was helped (or harmed), whether family poverty was 
reduced or left unchanged, and whether preventive health care practices and parents’ employ- 
ment prospects were improved. They urged that final judgments about the merits of Family 
Rewards and whether it had any place in public policy wait until the results of the evaluation 
were available. 13 

They also challenged the moral arguments against incentives, stressing that incentives 
were already used widely throughout society to influence the choices and activities of middle- 
class and wealthy people, including tax deductions designed to encourage home purchases and 
tax breaks to steer a multitude of business investment decisions in particular directions. 14 Fur- 
thermore, the program offered a substantial amount of extra resources to poor families, adding 
to — not competing with — other benefits that were already available to them. 


Evaluation 

The Family Rewards model was operated in a total of six community districts — two each 
from the Bronx, Brooklyn, and Manhattan. These six areas were chosen because they were 
among New York’s most persistently disadvantaged communities. At the same time, these 
communities were diverse in terms of their racial and ethnic composition (although most resi- 
dents are black or Hispanic/Latino), and in terms of the prevalence of social service and 
health organizations. 15 

The program was evaluated through a randomized controlled trial involving approxi- 
mately 4,800 families who applied for it, with 11,000 children (including elementary, middle, 
and high school students). The program could not serve all applicants, and participants were 
selected on a random basis. Through a lottery-like process, half of the applicant families were 
picked for Family Rewards and were offered the incentives (the program group), and half were 
assigned to a control group that was not offered the incentives. Differences between the pro- 
gram and control groups’ outcomes reflect estimates of the program’s impacts; statistically sig- 
nificant differences are those that are likely a result of the program rather than chance alone. 
Using such a process helped ensure that program effects estimated by the evaluation could be 
attributed with confidence to the intervention. 


13 New York City Office of the Mayor (2007). 

14 In a speech at a conference on CCT programs hosted by the Organization of American States in Septem- 
ber 2009, former Mayor Bloomberg said, “Financial incentives have already proved to be a powerful tool in so 
many areas. The federal government puts them in our tax code, through policies such as the mortgage interest 
tax deduction for homeowners. And the private sector uses them too, in the form of compensation packages. So 
why shouldn't local governments also harness the power of incentives?” See New York City Office of the 
Mayor (2009). 

15 For more background on these communities and the families that enrolled in the study, see Riccio et al. 
( 2010 ). 
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The evaluation used an extensive set of quantitative and qualitative data. This infor- 
mation includes administrative records on school outcomes, employment, earnings, public 
health insurance, and welfare and food stamp benefit payments obtained from various New 
York City and New York State agencies; two waves of a survey in which a subset of parents in 
the program and control groups were interviewed; a special survey of a subsample of high 
school students and their parents; program-related data on incentive payments (“reward pay- 
ments”) obtained from Seedco; and qualitative data obtained through in-depth interviews with a 
sample of program participants and through observations of staff carrying out program activities 
at Seedco and the demonstration’s Neighborhood Partner Organizations (NPOs) — local 
organizations in the designated community districts that were selected to assist in implementing 
Family Rewards. 


Findings in Brief 

During the three years that Family Rewards operated, the average participating family earned 
over $8,700 in rewards, or roughly $3,000 in each year. As the program designers intended, 
families earned rewards across a broad range of areas, allowing them to increase their average 
monthly income by 22 percent during the program period. This reward system led to statistical- 
ly significant reductions in poverty and, more important, to reductions in a number of material 
hardships, including not having enough food to eat. No restrictions were imposed on how fami- 
lies could spend their reward money, and parents used the reward money to pay for basic 
household expenses, some “extras,” and, in some cases, to save for college and pay for special 
lessons to help their children in school. However, the reductions in poverty and material hard- 
ship began to diminish after Year 3, when the program and its cash transfers ended. 

The program had some positive impacts on family behaviors and outcomes, but also left 
many important outcomes unchanged. In terms of children’s school progress, Family Rewards 
improved outcomes for ninth-graders who were perfonning relatively better academically than 
their peers when they entered the study. For example, it increased their attendance rate, credit 
accumulation, and rate of passing New York State Regents exams. 16 For the better-prepared 
subgroup of ninth-graders who were reading-proficient according to their eighth-grade stand- 
ardized tests (although not for those who were math-proficient), the early gains in school per- 
formance led to sizable increases in grade promotion and a higher graduation rate. A special 
study of teens’ time use suggests that these positive effects were at least partly achieved by get- 
ting teens to spend more time on academic pursuits when they were not in school, and less time 


16 Regents exams are administered to all public high school students in New York State. Students must 
pass at least five tests in specified subject areas in order to graduate with a diploma recognized by the New 
York State Board of Regents, which sets standards and regulations for all public schools. 
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on social pursuits. 17 That study also found evidence that the program may have reduced certain 
problem behaviors and substance abuse among teens in the program. Although Family Rewards 
does not appear to have increased overall enrollment in college and other postsecondary institu- 
tions, it seems to have led to a shift among students in the reading-proficient subgroup toward 
attending college full time rather than part time, and a shift from attending two-year institutions 
to attending four-year colleges. 

In contrast, the program had no effect for students in the ninth-grade cohort who per- 
formed below proficiency levels on their eighth-grade standardized tests. Nor did it have any 
effects on educational outcomes for elementary and middle school students, either overall or for 
particular subgroups. 

A supplementary — and exploratory — analysis examined effects on school outcomes 
for young children who were the siblings of older students in the study. These children ranged 
from preschool age to kindergarten age when their parents enrolled in Family Rewards, and 
most were not expected to benefit directly from the education rewards during the period that the 
program operated. (They were 2 to 7 years old at the time of random assignment, and in grades 
3 through 6 in the sixth year of follow-up.) The program increased the proportion of these stu- 
dents scoring at the proficient level on their annual standardized tests in whatever grade they 
were in during Year 6 of the follow-up period. Although the effects are not large and the mech- 
anisms for them are not clear, the increased income that Family Rewards provided to these 
households may have benefited those younger children (not just older siblings or adults) during 
that early fomiative period in their lives. 

In the health area, early positive effects on visits to the doctor and on health status faded 
for the full sample. However, some suggestive evidence indicates that the incentives may have 
improved self-rated health outcomes for adults who said that they were in “fair or poor” health 
when they entered the program, and may have also increased (by a small amount) the likelihood 
that adults in better health would get treatment for certain medical conditions. Sizable increases 
in dental visits persisted for adults as well as children. 

Finally, the program produced small increases in training and educational credentials 
among adults and led to modest increases in employment in jobs that were not covered by or 
reported to the unemployment insurance (UI) system (such as private child care jobs). However, 
it had no overall effects on employment in Ul-covered jobs, and, according to subgroup anal- 
yses, led to small reductions in employment and earnings among more disadvantaged adults in 
the study. 


17 Morris, Aber, Wolf, and Berg (2012). 



Family Rewards 2.0 

As the early results of the Family Rewards study emerged, they informed initial insights about 
the possible strengths and limits of the original design. In light of that early evidence, and the 
expansion of CCTs internationally, CEO and MDRC decided to develop a revised version of 
the Family Rewards approach, dubbed “Family Rewards 2.0.” Among other changes, the new 
version included fewer incentives and a “family guidance” component to help families meet the 
conditions that would earn them rewards. The new program was launched as a federal Social 
Innovation Fund project in the Bronx, New York, and in Memphis, Tennessee, with a combina- 
tion of private and federal funds. 18 MDRC is conducting a comprehensive evaluation of that 
project, which operated from 2011 through 2014. Impact findings from that study will be avail- 
able by late 2016 and will show whether or not the new model (which is described more fully in 
Chapter 5 of this report) produced larger impacts across more outcomes for a broader range of 
participants. 19 


The Remainder of This Report 

The next three chapters in this report describe the design of and theory underlying the original 
Family Rewards program model, the operation of the program, and its impacts on income, pov- 
erty, and material hardship as well as on outcomes in each of the three key domains — chil- 
dren’s schooling, family health, and parents’ work and training. The final chapter provides a 
summary assessment of the program. It explores whether certain types of design modifications 
to the Family Rewards model might produce stronger results. 


18 The Social Innovation Fund is a program of the Corporation for National and Community Service that 
combines public and private resources to replicate and further test promising innovations to improve the lives 
of people in low-income communities. 

19 For early findings on the implementation of the revised program and families’ patterns of reward receipt, 
see Dechausay, Miller, and Quiroz-Becerra (2014). 
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Chapter 2 

Designing Family Rewards 


Opportunity NYC-Family Rewards represented an ambitious attempt to adopt the principles of 
international conditional cash transfer (CCT) programs and incorporate them into a new two- 
generation poverty-reduction initiative for very-low-income families living in some of the high- 
est-poverty communities in New York City. Although the first of its kind in terms of its com- 
prehensiveness and scope, the program was certainly not the first one in the United States to 
condition government transfer benefits on certain behaviors. One of the largest income transfer 
programs, the Earned Income Tax Credit (EITC), is an earnings supplement paid only to low- 
income families who are working, and the Temporary Assistance for Needy Families (TANF) 
program includes work requirements that condition welfare payments on parents’ efforts to pre- 
pare for and look for work. Family Rewards expanded the typical scope of conditionality by 
tying some income transfers to families to their children’s school progress and to parents’ and 
children’s receipt of preventive health care as well as to parents’ employment. 

Designing this program was a complex challenge. It had to incorporate a set of condi- 
tions that would be achievable by families so that they could earn sizable cash transfers (to re- 
duce immediate poverty and hardship), conditions that were verifiable , incentive amounts that 
would be potentially affordable if Family Rewards were scaled up as part of a government pro- 
gram, and incentives that would help families take steps to build their human capital. This chap- 
ter summarizes the design that emerged and the reasoning behind the choices that were made. 


The Theory Behind Family Rewards 

Family Rewards rested on the premise that financial incentives can encourage individuals to 
make short-term choices and take certain actions that may serve their best interests over the 
longer term. Furthermore, the extra income earned through incentive payments (“reward pay- 
ments”), if they are sufficiently large, can make it easier for families to take those steps to im- 
prove their human capital, while simultaneously helping to alleviate their immediate material 
hardships. 

The simple lack of income can impede families’ investments in their own futures . 1 Un- 
derstandably, poor families have more difficulty than better-off families in getting access to 
good schools and providing their children with educational materials and enrichment programs; 


A growing body of evidence suggests that very low income, material hardship, and financial strain may 
have negative causal influences on children’s life trajectories. For a review of evidence on the relationship be- 
tween family income and child development, see Duncan, Magnuson, and Votruba-Drzal (2014). 
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paying for tutoring for children who need extra help; affording reliable child care when parents 
work; getting paid for time off from work to take their children and themselves to medical or 
dental checkups; finding dentists who are willing to take Medicaid; and even paying for trans- 
portation to and from low-wage jobs or job interviews, health visits, and school activities. 

Some experts contend that parents who are struggling with poverty may have difficulty 
devoting mental and emotional energy to helping their children succeed in school and to ad- 
vance themselves economically. When financial resources are scarce, parents’ preoccupation 
with providing the basic necessities for their families and constant worry about making ends 
meet on so little income can make it more difficult for them to focus on and plan effective ways 
of escaping poverty . 2 

Some evidence suggests that low-income individuals may “discount the future” more 
substantially than higher-income individuals, meaning that they may not attach sufficient value 
to investments in education or health-related practices that can improve their longer-tenn eco- 
nomic security, health, and well-being because they may not fully recognize or believe in the 
future payoffs that might come from such investments . 3 In high-poverty communities, commu- 
nity or peer group nonns, fed by observations of persistent intergenerational poverty, may rein- 
force lower expectations. For young children in particular, the future is very distant, and posi- 
tive long-term outcomes may be too abstract to be a significant motivating force. 

Conditional cash transfers may help change the equation. In the face of more immediate 
and tangible rewards (a source of “extrinsic motivation”), people may take steps that serve their 
longer-tenn best interests, and perhaps develop new habits, even without fully recognizing or 
believing in the longer-tenn value of those short-term efforts. For example, with the right incen- 
tives, students may attend school more regularly or study harder; parents may focus more atten- 
tion on preventive health care for their families and making lifestyle choices that promote better 
heath; and parents may increase their efforts to find and keep jobs. Furthennore, if the rewards 
are sizable, as they were in Family Rewards, the extra resources can help make it more feasible 
for low-income people to undertake certain educational, health care, and work-related invest- 
ments in the short term. As resources accumulate from some activities, such as children’s school 


2 Mullainathan and Shafir (2013, p. 47) describe two related components of mental functioning that are af- 
fected by poverty: (1) cognitive capacity, "the psychological mechanisms that underlie our ability to solve 
problems, retain information, engage in logical reasoning, and so on,’’ and (2) executive control, "which under- 
lies our ability to manage our cognitive activities, including planning, attention, initiating and inhibiting ac- 
tions, and controlling impulses.” They emphasize that “scarcity affects both.” Mani, Mullainathan, Shafir, and 
Zhao (2013) write, "The poor must manage sporadic income, juggle expenses, and make difficult tradeoffs. 
Even when not actually making a financial decision, these preoccupations can be present and distracting. The 
human cognitive system has limited capacity. Preoccupation with pressing budgetary concerns leaves fewer 
cognitive resources available to guide choice and action.” 

3 See, for example, Fiszbein and Schady (2009) for a fuller discussion of this issue. 
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attendance and family members’ doctor visits, the extra money can help cover the costs of other 
activities and materials, such as the cost of school supplies, fees for enrichment programs, or 
tutoring for children; transportation to a free dental clinic; the dental checkup itself (if not free 
or covered by insurance); clothes or transportation for a job interview; or certain fees for an 
adult training program. The increased income through the cash transfers may also reduce some 
of the stress and cognitive constraints arising from poverty, leading to improvements in overall 
family functioning. Thus, the conditioned payments may function not only as financial induce- 
ments that stimulate action, but also as enabling resources that make it easier to be productive, 
and as income supplements that mitigate the difficulties associated with poverty. 4 


Eligibility for Family Rewards 

To be eligible for the program, families had to have at least one child in the fourth, seventh, or 
ninth grade, although once a family was enrolled, all school-age or younger children could par- 
ticipate in it. Students in these three grades are at or near the start of critical educational transi- 
tion years. For example, some research shows that children who fall seriously behind in school 
by the third or fourth grade have difficulty catching up later. 5 The designers of Family Rewards 
thus hoped that offering three years of education incentives for the fourth-grade target group 
would help them do better in that critical year and successfully weather the transition to the first 
full year of middle school, which typically begins in sixth grade in New York City. Similarly, 
rewards that were targeted to the cohort of seventh-graders were intended to help support and 
encourage those students to perform well in middle school through the end of ninth grade (the 
first year of high school), another critical transition stage. 

The families also had to be eligible for free school meals under the National Free 
School Lunch program. This federally funded program provides free lunches for children in 
families with incomes at or below 130 percent of the federal poverty level (and subsidized 
meals for other students with somewhat higher incomes). Enrollment in the free school lunch 
program was a proxy for determining a family’s income eligibility for the purposes of the 
demonstration. 6 The families also had to be living in one of the six designated high-poverty 
communities, and all parents and children had to be legal residents of the United States. 

A majority of the families (81 percent) who enrolled in Family Rewards were one- 
parent families at the time of random assignment, but they differed widely in a number of other 


4 Duncan, Magnuson, and Votruba-Drzal (2014) cite evidence suggesting that increasing poor parents’ in- 
comes may in and of itself help children achieve better school and life outcomes, and that the effects may be 
greatest when children are very young and their brains and other systems are developing rapidly. 

5 For a more recent discussion of this pattern and the importance in particular of early reading proficiency, 
see Feister (2013). 

6 See Riccio et al. (2010), page 36, for more details. 
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background characteristics. For example, over half of all families (57 percent) had only one or 
two children, but 43 percent had three or more. About 47 percent of the families were Hispan- 
ic/Latino, while most others (5 1 percent) were black, non-Hispanic/Latino. Just over half of the 
parents (53 percent) were employed, with about 40 percent working full time. About one-fourth 
(26 percent) had only a high school diploma or General Educational Development (GED) certif- 
icate, and about 14 percent had an associate’s or bachelor’s degree, while 40 percent had not 
completed high school and did not have a GED certificate. About 83 percent were U.S. citizens, 
while the rest (17 percent) were legal permanent residents. 


Types of Rewards 

Family Rewards offered a set of 22 different incentives during its first two years (some of which 
were discontinued in Year 3), ranging in value from $20 to $600 each per year. (See Table 2.1 
for a detailed list.) These rewards did not count as income that would affect other benefits the 
families might receive, such as TANF benefits, EITC payments, housing subsidies, Medicaid, 
the Children’s Health Insurance Program, (CHIP), 7 or Supplemental Nutrition Assistance Pro- 
gram (SNAP) benefits (formerly known as food stamps). 8 

Education Rewards 

Poor academic achievement and dropping out of school are widely understood to be 
associated with worse future employment outcomes and earnings, contributing to intergenera- 
tional poverty. 9 And, increasingly, as the U.S. labor market requires more critical thinking and 
specialized skills, postsecondary education has become a critical pathway to good jobs. 10 Con- 
sequently, succeeding in elementary, middle, and high school is one of the most important ways 
that children growing up in poor families can prepare for the future and avoid poverty in adult- 
hood. For this reason, children’s school perfonnance and academic achievement were a central 
focus of Family Rewards. 

In contrast to the limited scope of education incentives in other countries’ CCT pro- 
grams, which primarily targeted school enrollment and attendance, 1 1 Family Rewards included 


7 CHIP, also known as Child Health Plus in New York, targets uninsured children and pregnant women in 
families with incomes too high to qualify for Medicaid, but often too low for them to afford private coverage. 

Supplemental Security Income benefits were an exception. A waiver had been sought from the Social Se- 
curity Administration, but the request was not approved. The small number of families receiving such benefits 
were informed about other agencies in the community that could help them understand the implications of re- 
ward payments for their benefit amounts. 

9 Haskins and Sawhill (2009). 

10 Levy and Mumane (2004). 

"Fiszbein and Schady (2009). 
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Table 2.1 


Incentives Offered in Opportunity NYC-Family Rewards 


Activity 

Reward Amount 

Education incentives 


Elementary and middle school students 


Attends 95% of scheduled school days 
(discontinued after Year 2) 

$25 per month 

Scores at proficiency level (or improves) on annual 
math and English language arts (ELA) tests 

Elementary school students 

Middle school students 

$300 per math test; $300 per ELA test 
$350 per math test; $350 per ELA test 

Parent reviews low-stakes interim tests 
(discontinued after Year 1) 

$25 for parents to download, print, and 
review results (up to 5 times per year) 

Parent discusses annual math and ELA test results 
with teachers (discontinued after Year 2) 

$25 (up to 2 tests per year) 

High school students 


Attends 95% of scheduled school days 

Accumulates 1 1 course credits per year 

Passes Regents exams 

Takes PSAT test 

Graduates from high school 

$50 per month 
$600 

$600 per exam passed (up to 5 exams) 

$50 for taking the test (up to 2 times) 

$400 bonus 

All grades 


Parent attends parent-teacher conferences 

Child obtains library card ( discontinued after Year 2) 

$25 per conference (up to 2 times per year) 

$50 once during program 

Health incentives 


Maintaining public or private health insurance 
(discontinued after Year 2) 

For each parent covered 

If all children are covered 

Per month: $20 (public); $50 (private) 

Per month: $20 (public); $50 (private) 

Annual medical checkup 

$200 per family member (once per year) 

Doctor-recommended follow-up visit 
(discontinued after Year 2) 

$100 per family member (once per year) 

Early-intervention evaluation for child under 30 months 
old, if advised by pediatrician 

$200 per child (once per year) 

Preventive dental care (cleaning/checkup) 

Workforce incentives 

$100 per family member (once per year for 
children 1-5 years old; twice per year for 
family members 6 years of age or older) 

Sustained full-time employment 

$150 per month 

Education and training while employed at least 10 hours per 
week (employment requirement discontinued after Year 2) 

Amount varies by length of course, up to a 
maximum of $3,000 over 3 years 
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incentives tied to a much broader range of school-related behavior and achievement. These in- 
centives included rewards for high attendance rates, certain types of parental engagement, stu- 
dent perfonnance on standardized tests, and even a reward for getting a library card. The value 
of the rewards ranged from $25 per month for achieving a 95 percent attendance rate in elemen- 
tary and middle school to $600 for passing a high school subject area Regents exam. 

Reflecting the important role that parents can play in their children’s success in 
school, 12 the incentives in this domain were intended to encourage parents to become more fully 
engaged with their children’s education. In this way, Family Rewards differed from school- 
based incentives programs that only offer rewards directly to students, largely bypassing their 
parents. In Family Rewards, the education incentives for elementary and middle school students 
were paid entirely to the parents, and what children learned about the rewards depended on 
what their parents chose to tell them. In addition, the schools played no direct role in the pro- 
gram, and teachers would not know which of their students were in it unless the students them- 
selves or their families informed them. 

At the high school level, the focus shifted. Although some rewards were still paid to the 
parents, others were partially or completely paid to the students directly. This policy was based 
on the recognition that high school students increasingly take on responsibility for their own 
school and consumption behaviors, and so paying them directly would give them an immediate 
financial stake in their school perfonnance. 

Health Care Rewards 

Not surprisingly, a number of studies have shown that poor children tend to have worse 
health outcomes than their peers who are not poor. 13 In addition, studies of welfare recipients 
and other low-income populations suggest that health problems are a common impediment to 
steady work among the parents in low-income families. 14 To some extent, these outcomes may 
derive from inadequate preventive health care. 

For a variety of reasons having to do with a lack of resources, knowledge, information, 
or administrative hurdles posed by the health care system, many low- and moderate-income 
families do not get routine preventive health care. To some extent this problem results from 
their lack of health insurance coverage or gaps in that coverage. For example, some families 
who are eligible for public insurance do not realize they are eligible and remain uninsured. Oth- 
ers who know they are eligible may not follow through with the application process, which can 
be burdensome or confusing. Similarly, many families lose public insurance when they fail to 


12 See Haskins and Sawhill (2009) for a brief review of evidence. 

°Aber, Bennett, Conley, and Li (1997). 

14 Bloom (1997); Polit, London, and Martinez (2001); Zedlewski and Loprest (2001). 
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complete the necessary recertification process. Other low-income individuals who are working 
and are eligible to participate in employer-sponsored health insurance plans cannot afford the 
required copayments. 

Families who lack health insurance are understandably less likely to have a regular fam- 
ily physician and tend to rely on more costly hospital emergency rooms when they need care. 15 
However, even families with health insurance, particularly public health insurance, may not 
maintain good preventive care or follow up on recommended treatments — a problem that cuts 
across socioeconomic groups. (Notably, a number of health insurers have begun offering their 
subscribers financial incentives to go to the gym.) Low-income families who rely on public in- 
surance face the added challenge of finding doctors who will accept their insurance, which can 
be difficult because of low reimbursement rates and may discourage them from getting regular 
preventive care. Similarly, many dentists do not accept Medicaid, so getting preventive dental 
care may require low-income families to search for dental clinics that offer free care or that 
charge according to a sliding scale based on a family’s income. Parents who are working in 
low-wage jobs that do not offer paid personal or sick days may face an added opportunity cost 
in getting preventive care if they lose pay when they miss work for a doctor’s visit. 

Anticipating that these broad, nationally observed trends would be significant problems 
in the high-poverty communities that were targeted for Family Rewards, and following the 
practice of international CCT programs, the designers of Family Rewards placed a major em- 
phasis on preventive health care. They attached rewards to the maintenance of health insurance 
as well as to getting regular medical and dental checkups. 16 The rewards ranged in value from 
$20 per month per adult for maintaining public health insurance to $200 per family member for 
an annual physical. The designers hoped that sustained insurance, better preventive care, and 
quicker responses to emerging health problems would help to reduce poverty by removing bar- 
riers to steady employment among parents and by minimizing school absences and improving 
school perfomiance among children. 

The Work and Training Rewards 

Work does not guarantee an escape from poverty, as the growing number of “working 
poor” families attests. 17 Skills deficits and wage stagnation are important impediments to eco- 


15 See Polit, London, and Martinez (2001). 

16 To qualify for rewards for annual physicals, participants were expected to present doctors with an age- 
appropriate “preventive care checklist form,” specially designed for Family Rewards, that identified a set of 
common health conditions that doctors should explore or screen for in any thorough annual physical examina- 
tion. Separate forms were created for parents and children. See Riccio et al. (2010) for further information. 

I7 ln 2005, over 46 percent of poor households in New York City were “working poor” — that is, they had 
incomes below the poverty level despite the fact that the head of household worked at least part of the year. See 
New York City Center for Economic Opportunity (2006). 
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nomic advancement. But it is also true that, for most families, it is impossible to improve family 
income and escape poverty without working. Family Rewards could increase family income 
through the direct cash transfers themselves, but only temporarily. Sustained reductions in child 
and family poverty after the payments end would require that parents maintained regular em- 
ployment. The program thus included a work component that was intended to promote steady, 
full-time employment and the acquisition of skills to help participants qualify for better-paying 
jobs. The designers also expected that a cash transfer program that included a component that 
explicitly supported and encouraged families’ pursuit of self-sufficiency could have broader 
appeal across the political spectrum than one that did not. This type of component is absent 
from most other CCT programs and thus holds special interest to the international community. 

Family Rewards offered parents a reward for sustaining full-time employment. In order 
to earn this reward, a participant was required to work at least 30 hours per week for six out of 
every eight weeks. Parents who worked the minimum amount received $300 every two months, 
or up to $1,800 per year. By increasing the payoff to work, the reward was intended to create an 
incentive for parents to find full-time jobs, to move from part-time into full-time work, or to 
stay in full-time employment. For a parent working 40 hours per week at $8 per hour, for exam- 
ple, the reward would effectively increase her net wage by 1 1 percent, to $8.90 per hour. 

Parents could earn another $300 to $600 every two months, up to a total of $3,000 dur- 
ing the program period, if they completed an approved education or training course. In order to 
qualify for the education and training rewards during the first two years of the program, parents 
had to participate in approved education or training activities while working at least 10 hours 
per week. Because the receipt of these rewards was so low in Years 1 and 2 (less than 2 percent 
of eligible adults earned these rewards), the minimum work hours requirement was discontin- 
ued in Year 3. 


A Complex Design Challenge 

In establishing the scope and value of the incentives, the program designers sought to balance a 
variety of theoretical, practical, and political concerns (as described in Box 2.1). They included 
a broad range of rewards to create opportunities to assess which incentives might be the most 
effective. In addition, they sought to give families many different ways in which to earn money 
and to avoid attaching overly large amounts of money to any particular activity or outcome. Af- 
ter reviewing early evidence of impacts, several rewards were discontinued for the third year to 
simplify the program, lower its costs, and make it easier to replicate should it prove to be suc- 
cessful overall. 

The program allowed families to receive cash rewards totaling several thousand dollars 
per year over a three-year period. The actual amounts that families received depended on the 
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Box 2.1 

The Family Rewards Incentives: Overall Design Considerations 

In developing the rewards schedule, the Family Rewards design team (CEO, MDRC, 

and Seedco) was guided by the following key principles: 

1. Incentives should be attached to activities and achievements that represent invest- 
ments in human capital development. 

2. The conditions for incentive payments should be achievable with a reasonable level 
of effort. 

3. The incentives should not be tied to activities that are not generally available or rea- 
sonably accessible to participants (such as attendance in early Head Start programs). 

4. To the extent feasible, more money should be attached to conditions that are more 
challenging to meet. (For example, students’ passing standardized tests and parents’ 
sustaining full-time work should earn more than a parent’s attendance at a parent- 
teacher conference.) 

5. The activities and achievements that earn payments should be verifiable in ways that 
are practical, timely, and resistant to fraud. 

6. Incentives for children’s school performance should avoid putting undue pressure on 
students or putting them at risk of abuse if their families lose out on getting extra 
money because of their poor performance. 

7. The amounts for any given activity should be substantial enough for families to take 
the offer seriously. 

8. The amounts for any given activity — and overall — must not be so high as to ap- 
pear extremely unreasonable by policymakers and the public, and, hence, politically 
unsustainable. 

9. High compliance with the conditions by all family members across all domains 
should yield a substantial total cash transfer (in the range of 25 percent of family in- 
come in the absence of the program). 

10. A broad range of incentives should be included to give families many different ways 
to earn rewards — and to earn meaningful amounts of money within each of the 
three domains. 


number and particular type of rewards they earned. (Some rewards carried higher payments 
than others.) Larger families could earn higher payments because they could earn education and 
health rewards for each child. 
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Mutually Reinforcing Rewards 

Although each of the three components targeted specific problems that contribute to 
long-tenn and intergenerational poverty, the Family Rewards model assumed that sustained 
achievements in any one of these areas may be aided by progress in the others. For example, 
children may make more progress in school if their health care is improved and efforts are made 
to catch and address health problems early. Their health and education may benefit if they grow 
up in a household that has increased economic resources at its disposal, for which parents’ sus- 
tained employment is critical. Children’s health as adults may also be influenced by their educa- 
tion, which can affect their understanding of good health practices and healthy lifestyle choices. 
In sum, the model assumed that the health rewards were not just important to health outcomes, 
that the education rewards were not just important to education outcomes, and that the work 
rewards were not just important to work outcomes. For all these reasons, combining all three 
components into a single, two-generation package was considered to be more powerful than 
focusing on any one or two of these components alone. In addition, as noted previously, the ex- 
tra income from all three components, which would serve the program’s immediate poverty 
reduction goal, could have important beneficial effects on child outcomes simply by reducing 
the hardships of poverty. 


No Case Management 

By design, Family Rewards included no case management. This feature meant that no provision 
was made for staff to develop action plans that could address barriers in participants’ lives or to 
intervene in personal crises that may have made it difficult for participants to succeed in the 
program. Family Rewards also made no provision for staff to follow up with participants on 
their individual progress in meeting their goals or getting the services they needed, or to inter- 
vene directly with service providers on behalf of individual families — for instance, by help- 
ing to arrange tutoring for children; offering assistance finding child care; or making service 
referrals for job search assistance, occupational skills training, or doctor visits. Family Rewards 
also did not provide any direct services, such as tutoring, test preparation, job search classes, 
basic educational instruction, or occupational training. 

The program’s designers excluded these fonns of assistance and services for four main 
reasons. First, they wanted to test the power of the cash incentives alone. Second, in contrast to 
participants in CCT programs in poorer nations, they expected that many families would have 
access to services through other programs in the community (and, because of Family Rewards, 
more reason to take advantage of them), and that Family Rewards should not duplicate those ser- 
vices. Third, they reasoned that if the New York City model could succeed without those extra 
elements, it would be easier and less expensive to expand the scale of the program as an ongoing 
policy. And fourth, they hoped that the program would be less burdensome to families if they 
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were not required to have regular appointments with staff while also trying to balance work and 
family obligations. Thus, just as low-income workers need not take part in services in order to 
benefit from the EITC, it was possible for Family Rewards participants to be fully engaged with 
the program without meeting with program staff (except to pick up their reward verification 
coupons). 

Still, the designers recognized that many families would need at least some guidance on 
where they could find the kinds of services and assistance that might enhance their success in 
the program — for example, where they could find tutoring or after-school programs, dental 
clinics, job search programs, and job training programs. For that reason, the program included 
an informational component through which Seedco and the participating community organiza- 
tions were expected to help educate families about relevant resources that are available in the 
community, such as by disseminating written resource guides listing agencies that might be ap- 
propriate for parents to contact, and by reminding families about those resources in optional 
workshops and in marketing materials. 
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Chapter 3 

Delivering Cash Rewards 


As the first comprehensive conditional cash transfer (CCT) program in the United States, Fami- 
ly Rewards was breaking new ground, and given the model’s many facets and its many opera- 
tional demands — recruiting, informing, verifying, and paying participants on a trial basis — its 
operational success could not be taken for granted. In the end, the program passed a basic feasi- 
bility test: the operators successfully transferred substantial amounts of money to low-income 
families, with all payments tied to a comprehensive set of conditions, and virtually all partici- 
pating families earned and received rewards. This chapter reviews how the program was operat- 
ed, how the participants viewed and responded to it, the families’ patterns of reward receipt, and 
what the program cost. 1 


Operating the Program 

The implementing organizations succeeded in operating all the major program systems and pro- 
cedures that the Family Rewards model required, although it took until the second year for the 
program to function as envisioned. Program operations remained strong in the third and final 
year, which ended, as originally planned, in August 2010. The program successfully distributed 
$20.6 million in reward payments over the full course of the three-year program. 

The Delivery Structure 

Seedco, the main implementing agency, assembled a network of local organizations in 
the designated community districts to assist in implementing Family Rewards. Called “Neigh- 
borhood Partner Organizations” (NPOs), these agencies recruited and enrolled eligible families 
into the research sample and served as the face of the program in the communities. 2 They pro- 
vided ongoing customer service (not case management) to participants who requested assistance 
with, for example, submitting claims for the rewards or getting infomiation about other services 
in the community. NPOs also conducted informational workshops on how to earn and claim 
rewards in each of the domains in which the incentives were offered. Seedco maintained a tele- 
phone helpline and a website to provide additional information to families. 


For a full description and analysis of the sample recruitment process, program implementation experi- 
ence, and participants’ patterns of reward receipt and views of and experiences in the program, see Riccio et al. 
(2010); Greenberg, Dechausay, and Fraker (2011); and Riccio et al. (2013a). 

2 These organizations are Urban Health Plan and BronxWorks (formerly Citizens Advice Bureau) in the 
Bronx; Brownsville Multi-Service Center and Groundwork, Inc., in Brooklyn; and Catholic Charities and Un- 
ion Settlement Association in Manhattan. 
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Seedco was responsible for verifying participants’ eligibility for reward payments. Ef- 
forts were made to minimize the steps that families had to take to establish their compliance 
with reward conditions. This was easily achieved in the domain of children’s education, for 
which Seedco obtained electronic data files from the New York City Department of Education 
to measure attendance and performance. Verification for all education rewards based on these 
benchmarks was entirely automated, requiring no action on the part of families to claim them. 
For other behaviors and accomplishments, families manually filled out special coupons with 
appropriate documentation verifying their compliance with the required conditions (for exam- 
ple, showing they had received an annual physical or worked full time), and they submitted 
these materials to Seedco to claim their rewards. 3 

Once Seedco verified that families had earned rewards, it initiated a process of transfer- 
ring payments electronically into participants’ newly opened or existing bank accounts or, if 
they preferred, onto stored value cards (which are prepaid cards, like gift cards). To provide 
families with a safe banking option, New York City officials worked with several banks and 
credit unions to develop special “Opportunity NYC accounts” that carried no fees and came 
with debit cards that were impossible to overdraw. The reward payments were made every two 
months, and families could access the money at any time through any automatic teller machine 
(ATM). 


Recruiting Families 

Program operations began with the start of the new school year in September 2007. To 
ensure that the program reached a broad cross-section of children, not just the most motivated 
and active, potentially eligible families who were living in the targeted communities were iden- 
tified from lists of students in the free school lunch program maintained by the New York City 
Department of Education. Seedco and the NPOs then attempted to recruit a representative group 
of those families through mailings, phone calls, and home visits, inviting them to apply to be in 
the study. Those who agreed were randomly assigned to the program or control group. 


Families’ Understanding of the Rewards 

Because Family Rewards was an “incentives-only” intervention, it was essential that families 
understood what conditions they would have to meet in order to earn the rewards. Thus, the 
program operators had to place a great deal of emphasis on marketing the incentives to the fami- 
lies and educating them about the requirements. They did this by creating a special website for 
participants; sending letters, flyers, and postcards to remind them about particular activities that 
would earn rewards (for example, upcoming tests that the children had to take in school); send- 


3 See Riccio et al. (2010) for more detail on these verification forms and processes. 
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ing earnings statements after each payment period specifying how much they had actually 
earned and reminding them of the rewards they could earn; conducting special informational 
workshops on school, health, and employment topics; and holding social events (such as holi- 
day parties and summer barbeques) that offered further opportunities to remind families about 
the program. 

• Parents understood the program’s general offer and purposes, but 
many were confused about the details. 

Survey and in-depth qualitative data reveal that parents possessed a good general 
awareness of the incentives, but their understanding was fuzzy in some places. For example, 
they tended to think that many school-related activities and behavior qualified for incentive 
payments when in fact only some of those activities and behaviors qualified. This confusion 
may indicate that participants believed that positive behavior in general would be rewarded. 
Such a misperception could have positive effects on families, but it shows that knowledge of the 
program was often imprecise. 

• Younger children’s understanding of the education incentives varied 
widely, in part according to how much their parents wanted them to 
know. Special marketing efforts were required to build awareness 
among high school students. 

Qualitative data suggest wide variation in the level of understanding that children and 
teenagers in the program may have had about the rewards. While some parents discussed the 
program in detail with their elementary or middle school-age children and viewed the rewards 
as another tool to motivate their children, others provided only limited information because they 
did not want to emphasize money as a reason for achievement. High school students, who could 
receive some incentive payments directly, were the target of independent marketing efforts. 
However, these efforts were not undertaken until the second year of operations, after it became 
clear that many of the participating high school students did not fully understand — or believe 
— that they could earn money for their school attendance and performance. 

Parents generally saw great value in the program as a support for their children’s educa- 
tional progress — and, indeed, many viewed it primarily as a program for the benefit of their 
children. Yet, they typically did not know how they could improve their children’s school per- 
formance beyond providing general encouragement. (See Box 3.1.) 
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Box 3.1 

How Families Responded to the Education Incentives 
in Family Rewards: A Qualitative Perspective 

As part of the evaluation of Family Rewards, MDRC conducted an in-depth qualitative 
study of the variety of ways that parents and children interacted with each other in rela- 
tion to the incentives. Unlike other educational incentives programs across the country, 
Family Rewards relied heavily on parents to explain the program to younger children and 
to find ways of supporting their children’s learning in school. While parents received 
incentive payments for their younger children’s activities in Family Rewards, high 
school students received incentive payments directly and so were more directly exposed to 
the program. Key findings include the following: 

• Most parents and children embraced the broad goals of Family Rewards, 
viewing the program as an opportunity for children and an investment in their 
academic future. It was not clear when the study began how families would view 
the program, but parents and children believed that Family Rewards was a worth- 
while idea because they believed it had the potential to support children’s academic 
performance and their success over time. 

• Many parents did not know how to improve their children’s educational per- 
formance beyond offering general encouragement. Many parents needed assis- 
tance in identifying additional strategies that they could use to help support their 
children’s learning, although some parents used reward payments for after-school 
activities or tutoring. 

• Many parents were reluctant to discuss the incentives with younger children, 
who had only a limited knowledge of the program as a result. High school stu- 
dents were much better informed. Some parents did not want to put financial pres- 
sure on their younger children, while others found creative ways to talk about incen- 
tives with them. Most likely as a result of the program being marketed directly to 
them, high school students were very aware of the program, and their parents re- 
ported that they were more likely to remind their older children of tasks that had to 
be completed in order to receive rewards. 

• Family Rewards payments helped strengthen some better-prepared high 
school students’ belief that they were “on track” to graduation, college, and a 
better future, which reinforced their motivation. Some highly motivated and 
generally proficient high school students used the rewards to save for college and to 
pay for educationally enriching experiences. 


SOURCE: Greenberg, Dechausay, and Fraker (2011). 
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Families’ Receipt and Use of the Cash Transfers 

Virtually all families earned at least some rewards during the three program years, and, as 
shown in Table 3.1, 89 percent earned at least one reward in Year 3 (when fewer rewards were 
offered). 


Table 3.1 


Summary of Rewards Earned by Families 


Outcome 

Year 1 

Year 2 

Year 3 

Years 1, 2, and 3 
Combined 

Family earned at least one reward (%) 

99.5 

98.0 

89.1 

99.6 

Education reward 

96.4 

91.5 

82.5 

97.9 

Flealth reward 

95.2 

94.2 

72.7 

98.0 

Workforce reward 

42.2 

42.1 

41.9 

53.2 

Among families who earned a reward in a specified 





period, average reward amount earned ($) 

3,153 

3,196 

2,700 

8,707 

Average reward amount earned, by domain 1 ($) 





Education 

1,450 

1,477 

1,394 

3,983 

Flealth 

1,224 

1,247 

879 

3,039 

Workforce 

1,359 

1,438 

1,475 

3,376 

Distribution of average reward amount earned 1 ’ (%) 





$1 - $99 

0.6 

0.6 

0.3 

0.2 

$100 -$499 

4.7 

5.2 

7.1 

1.1 

$500 - $999 

7.9 

7.2 

9.7 

2.1 

$1,000 -$2,999 

39.6 

37.9 

43.7 

11.2 

$3,000 - $4,999 

29.7 

31.3 

28.2 

14.5 

$5,000 - $6,999 

12.1 

13.2 

9.1 

13.6 

$7,000 or more 

5.4 

4.6 

1.8 

57.4 

Family picked up coupon book (%) 

89.8 

86.9 

87.0 

96.0 

Sample size 




2,377 


SOURCE: MDRC calculations using Seedco's Family Rewards program data. 

NOTES: Sample sizes may vary because of missing values. 

Rounding may cause slight discrepancies in calculating sums. 

The first program year covers September 2007 through August 2008; the second program 
year covers September 2008 through August 2009; and the third program year covers September 
2009 through August 2010. 

a Reward amounts are calculated among families who earned rewards for each domain during 
the specified period. 

b The maximum amount earned in Year 1 was $13,235; in Year 2, it was $12,525; and in Year 
3, it was $10,425. 
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• Overall, families earned a substantial amount of reward money — an 
average of over $8,700 for all three years combined. 

Reward amounts averaged over $3,100 during each of the first two years and $2,700 in 
the third year (when several rewards were discontinued). A majority of families — approxi- 
mately 57 percent — earned at least $7,000 over the life of the program. The top 20 percent 
earned more than $13,000 in reward money. 

To put these amounts in perspective, the federal poverty level for a family of three (for 
example, a single parent with two children) in 2009 (roughly midway through the program pe- 
riod) was $18,310. Thus, families of that size and income level who received $3,000 in reward 
payments in a year would increase their annual incomes by about 16 percent. Similarly sized 
families with incomes below half of the poverty level (or below $9,155 for the example cited 
above), used here as a measure of “severe poverty,” would boost their incomes by at least 33 
percent. Or, put differently, a reward amount of $3,000 would add about 21 percent to the total 
wages ($14,560) of a single parent who was paid $8 per hour for working 35 hours per week for 
an entire year. 

• Larger and relatively more advantaged families earned more cash 
transfers. 

Compared with other families, those who received the most money in reward payments 
(for example, those in the top 20 percent of families in terms of total reward dollars received) 
were larger families (giving them more opportunities to earn rewards) and tended to be less dis- 
advantaged. For example, the parents were more educated, more likely to be employed, and 
more likely to be married, and the families were less likely to be receiving government transfer 
benefits. In addition, in-depth interviews suggest that parents who were top earners may have 
been better organized, better able to handle the verification procedures associated with the pro- 
gram, and more likely to track their families’ performance against the conditions they needed to 
meet in order to earn rewards. 

Most reward money came from the education domain, accounting for 45 percent of the 
$20.6 million spent on reward payments over the full course of the program. Health care re- 
wards accounted for 34 percent of total payments, and workforce rewards (primarily for full- 
time work rather than education or training) accounted for 21 percent. 4 Virtually 100 percent of 
families earned at least one education reward and one health reward, while about 53 percent 
earned a workforce reward. 


4 Riccio et al. (2013a), 47-48. 
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• Parents used the reward money to pay for basic household expenses, 
some “extras,” and, in some cases, to save for college and pay for special 
lessons to help their children in school. 

Family Rewards imposed no restrictions on families’ access to their reward money or 
how they could spend it, and throughout the program families used the extra money in a variety 
of ways. Common uses included paying for basic living expenses, paying off bills, paying for 
school-related supplies or activities, buying electronic goods, saving for the future, and covering 
special recreational outings for the family, sometimes as a reward for school accomplishments. 
For many families, celebration of accomplishments took the form of spending time together on 
leisure activities, like eating out, going on a trip, or seeing a movie that would otherwise have 
been prohibitively expensive. 

Higli school students received substantial amounts of money in their own bank accounts 
for meeting education-related conditions, but parents exercised varying degrees of control over 
how much access students had to their rewards. The vast majority of parents who were inter- 
viewed for the 42 -month survey (72 percent) said that their high school-age children had to ask 
them for pennission to spend the money. Only 17 percent gave their children freedom to spend 
the money as they wished, and 9 percent did not allow their high school-age children to spend it 
at all. Despite the sizable money transfers into students’ own accounts, the program did not in- 
crease parent-teenager conflict, a problem that some observers feared. In addition, a companion 
study found that Family Rewards may have reduced certain troublesome behaviors among the 
teenagers, such as aggression and substance abuse. 5 

• During the final year, staff began to focus on an “exit strategy” to pre- 
pare families to cope with the ending of the reward payments. 

Given the relatively short period of the program and the fact that families would be ex- 
iting in the wake of the Great Recession, most participants were likely to see their income drop 
as the program came to an end. The staff tried to help participants prepare for this “income 
cliff’ by encouraging them to increase their labor market earnings and adjust their consump- 
tion patterns. 

In-depth interviews with a sample of participants suggest that families reacted to the 
end of the program with acceptance. They expressed gratitude for having had the experience, 
but some were doubtful that they could replace the lost income with earnings from employment. 


5 See Morris, Aber, Wolf, and Berg (2012). The analysis examined effects on a variety of delinquent be- 
haviors, aggressive behaviors, and drug use (including reported drug use among friends, which may be less 
vulnerable to reporting bias). Relative to their control group counterparts, teenagers in the program group re- 
ported substantial reductions across many of these outcomes. 
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Some expected to draw more on savings that they had accumulated during the program, and 
some expressed an intention to go back to school or try to increase their wage earnings. 


Program Costs 

Counting the total value of the reward payments and the expense to operate the program, Fami- 
ly Rewards cost an average of $13,093 per family (in 2014 dollars) for the entire three-year op- 
erating period, or approximately $4,364 per family per year. 6 When these costs are averaged per 
participant, taking all parents and children separately, Family Rewards cost about $3,798 per 
family member for the entire program period, or $1 ,266 per family member per year. 7 

Expenses to administer the program were incurred primarily by Seedco and the NPOs 
that were the face of the program in the local communities. For Seedco, these costs covered 
such activities as developing a payment tracking system, processing administrative records to 
determine whether automatically verified rewards had been earned, creating coupon books for 
reward payments that required families to submit documents proving that they had earned given 
rewards, verifying that requirements for coupon payment rewards were met, maintaining up-to- 
date bank account information to make sure payments were disbursed to the correct accounts, 
issuing earnings statements each payment period to mail to families, creating and maintaining a 
helpline to answer questions, making payments to families who earned rewards, marketing the 
program, performing general program management, and overseeing the NPOs. For the NPOs, 
costs were incurred for conducting program orientations for participants, providing refresher 
sessions, and organizing various social events and workshops. The NPOs were also responsible 
for distributing coupon books and providing general customer service. 8 

The cash transfers themselves (on average, more than $8,700 per family — or $8,900 in 
2014 dollars — over the course of the intervention) account for about 68 percent of program 
costs, with administrative expenditures accounting for the remainder (32 percent). 9 Put differ- 
ently, as a small-scale demonstration project, it cost Family Rewards about 47 cents in operating 
expenses per each dollar of rewards transferred to participants. This amount is high compared 
with ongoing, mainstream government transfer programs, such as Temporary Assistance for 


6 See Miller and Deitch (2016), available at www.mdrc.org, for full details on the cost estimation method- 
ology and results. 

7 These costs include sample recruitment, program start-up, and program wrap-up costs as well as 
the ongoing operation of the program. Some research costs incurred by the administrators of the program are 
included, but these are quite limited. See Miller and Deitch (2016) for a full discussion of findings from the 
cost analysis and how costs were estimated. 

8 See Riccio et al. (2010) and Riccio et al. (2013a) for further details on program operations and on the di- 
vision of responsibilities between Seedco and the NPOs. 

9 “ Administrative expenditures,” “administrative costs,” and “non-reward costs” are used interchangeably 
in this report. 
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Needy Families (TANF), the Supplemental Nutrition Assistance Program (SNAP), 10 or Medi- 
caid, which operate at a vastly larger scale and thus enjoy large economies of scale. Moreover, 
Family Rewards became more efficient over time. In the final year, it cost 34 cents per dollar of 
reward payments, although the number of rewards was also reduced. 

If Family Rewards were run at a larger scale, it would most certainly get to a point 
where the non-reward cost per reward dollar paid would decrease substantially. Just for the sake 
of illustration, if it were as efficient as TANF, for instance, administrative costs would be 1 7 
cents per dollar of reward payments. Applying this hypothetical estimate, and using the same 
average dollar value of reward payments, total administrative costs would be $1,486 per family, 
and overall program costs would be $10,386 per family — compared with $4,364 and $13,093, 
respectively, incurred during the demonstration. 


10 SNAP was formerly the Food Stamp Program. SNAP benefits are still often referred to as “food 
stamps.” 
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Chapter 4 

The Impact of Family Rewards on 
Poverty, Hardship, and Human Capital 


The long-term goal of Family Rewards was to reduce poverty across generations by promoting 
positive health care behavior, educational achievement, and employment — or “human capital 
development” — among low-income children and their parents. The short-term goal was to re- 
duce current poverty and material hardship through the program’s direct cash reward payments, 
or conditional cash transfers (CCTs), and through any immediate increases it could generate in 
parents’ earnings from employment, which could also boost Earned Income Tax Credit (EITC) 
payments. Overall, the program was more successful in achieving its short-term goal than its 
longer-term objectives. However, it did improve some human capital outcomes for some partic- 
ipants, including high school graduation rates. 


Current Poverty and Hardship 

Family Rewards succeeded in recruiting a population that continuously struggled with very low 
income and material hardships, as control group outcomes on a variety of measures attest. For 
example, three and a half years after they entered the study, control group parents reported an 
average family income from all sources of only $1,620 per month (or $19,440 on an annual ba- 
sis), as shown in Table 4.1. This finding indicates that about 68 percent of families were at or 
below the federal poverty level, with 27 percent having incomes below 50 percent of the pov- 
erty level — in what experts commonly consider “severe poverty.” Moreover, 44 percent were 
not able to pay their full rent or utilities bills at some point in the prior year or had other hous- 
ing-related hardships, and about 20 percent reported “sometimes” or “often” not having enough 
food to eat. Clearly, their needs were great, and those needs were exacerbated by New York 
City’s high cost of living. 

Effects on Current Poverty and Hardship 

• Family Rewards reduced families’ current poverty and economic hard- 
ships, including difficulties securing enough food and some housing- 
related hardships. 

Overall, when the program was operational and families were earning rewards, it was 
quite successful in achieving its short-term poverty reduction goal. For example, counting the 
value of the reward payments, it boosted self-reported average monthly household income for 
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Table 4.1 


Impacts on Selected Outcomes Measuring Poverty, Material Hardship, 
and Banking Through the Final Program Year or Early Post-Program Period 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Income and novertv 

Household income during Year 3 
(including Family Rewards payments) 

Average monthly income ab ’ c ($) 

1,973 

1,620 

353 *** 

0.000 

Annual income at or below federal poverty level db (%) 

56.0 

68.2 

-12.2 *** 

0.000 

Annual income less than 50% of federal 
poverty level a ' b (%) 

16.3 

27.4 

j *** 

0.000 

Household income during early post-program period 
(excluding Family Rewards payments) 

Average monthly income a ' b,c ($) 

1,700 

1,620 

79 * 

0.093 

Annual income at or below federal poverty level db (%) 

66.2 

68.2 

-2.0 

0.309 

Annual income less than 50% of federal 
poverty level a ' b (%) 

25.9 

27.4 

-1.5 

0.499 

Material hardship (%) 

Family "sometimes" or "often" did not have enough 
food to eat in past month 

18 -month survey 

14.8 

22.1 


0.000 

42-month survey 

15.3 

20.7 

_5 4 *** 

0.002 

Family usually did not have enough money to make 
ends meet at end of month 

18 -month survey 

34.1 

41.8 

j-j g *** 

0.000 

42-month survey 

35.4 

41.0 

-5.6 *** 

0.009 

Family did not pay full rent or mortgage in past year d 

18 -month survey 

39.0 

41.5 

-2.5 

0.245 

42-month survey 

40.0 

44.1 

-4.2 * 

0.061 

Parent agrees "strongly" or "somewhat" that current 
financial situation is "better than last year" 

18 -month survey 

62.7 

44.5 

18.3 *** 

0.000 

42-month survey 

51.4 

46.6 

4.8 ** 

0.034 

Banking and savines (%) 

Parent currently has any bank account 

18 -month survey 

73.3 

51.8 

2 \ 5 *** 

0.000 

42-month survey 

64.0 

46.6 

17 5 *** 

0.000 

Parent cashes check at check casher at least 
once a month 

18 -month survey 

29.2 

36.5 


0.000 

42-month survey 

29.2 

31.5 

-2.3 

0.260 


(continued) 
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Table 4.1 (continued) 


Outcome 

Program 

Group 

Control Difference 
Group (Impact) 

P-Value 

Family has any savings 1- 





18 -month survey 

25.7 

16.3 


0.000 

42 -month survey 

24.6 

16.8 

J g *** 

0.000 

Family's average savings exceed $50(T 





18 -month survey 

14.1 

8.0 

6.0 *** 

0.000 

42 -month survey 

12.5 

9.2 

3 2** 

0.023 

Parent borrows cash from family or friends 





(42 -month survey only) 

47.3 

52.5 

-5.2 ** 

0.021 

Sample size, 18-month survey (total = 2,060) 

- 16*1 




Sample size, 42 -month survey (total = 1,982) 

■■Hill 





SOURCE: MDRC calculations using data from the Family Rewards 18-month and 42-month surveys. 

NOTES: Sample sizes may vary because of missing values. 

A two-tailed t-test was applied to differences between outcomes for the program and control 
groups. The p-value indicates the likelihood that the difference between the program and control 
groups arose by chance. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 
percent; * = 10 percent. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 
assignment characteristics of sample members. 

Rounding may cause slight discrepancies in calculating sums and differences. 

See Riccio et al. (2010) and Riccio et al. (2013a,b) for more details on the measures and 
calculations used in this table. 

The items in the poverty, material hardship, and banking sections of the 42-month survey were 
administered to a random subsample of the survey respondents. 

The early post-program period refers approximately to the first six months since the time the 
program ended, which is when most respondents to the 42-month survey completed their interviews. 

a Monthly household income amounts equal to or greater than $10,000 were excluded from this 
calculation. 

b Annual household income is calculated by multiplying by 12 the respondent's income in the 
month prior to the survey interview. The federal poverty level was calculated based on annual income 
(monthly income multiplied by 12) and the household size at the time of the survey, applying the 
applicable poverty level threshold depending on when a respondent was interviewed. 

c The Year 1 income measures reported on the 18-month survey are within 3 percent of the 42- 
month income measures reported here. 

d Only about 4 percent of the 42-month survey sample owned an apartment or a house at the time 
of the survey. 

e About 6 percent of the 42-month survey sample is excluded from this analysis because of missing 
data. 
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the program group by $353 in Year 3, an improvement of about 22 percent relative to the con- 
trol group’s average monthly income of $1,620, as shown in Table 4.1. (See Box 4.1 for guid- 
ance on how to read the impact tables in this report.) This extra income reduced the proportion 
of families living at or below the federal poverty level by 12 percentage points relative to the 
control group rate of 68 percent. 1 The program also cut the proportion of families who were liv- 
ing in severe poverty by 1 1 percentage points (or nearly 41 percent). 


Box 4.1 

How to Read the Impact Tables in This Report 

In the context of this evaluation, an “impact” is a measure of the effect of the Family 
Rewards program, represented by the difference in outcomes between those families 
who were randomly assigned to the program group and those who were randomly as- 
signed to the control group. The “Difference” column in each impact table shows the 
differences between the two research groups’ outcomes — that is, the program’s es- 
timated impacts on the outcomes. The p-value shows the probability that this differ- 
ence, or impact, arose by chance, rather than as a result of the program; the higher the p- 
value, the more likely that the difference occurred by chance. The number of asterisks 
indicates whether the impact is statistically significant at the 1 percent (***), 5 percent 
(**), or 10 percent (*) level — that is, whether there is less than a 1, 5, or 10 percent 
likelihood that the difference arose by chance. 


Most of this poverty reduction is attributable to the cash transfers that families received, 
rather than to increased earnings from jobs. Once Family Rewards ended and the transfers were 
no longer available, the average income of families in the program group no longer differed 
substantially from that of the control group. 

The extra income that Family Rewards participants received during the program period 
helped those families reduce a variety of material hardships, also shown in Table 4.1. For ex- 
ample, the proportion of families experiencing food insufficiency (meaning that parents indicat- 
ed in survey interviews that their families “sometimes” or “often times” did not have enough to 


'in this study, income and poverty estimates include self-reported monthly cash income plus the cash val- 
ue of benefits from the Food Stamp Program, now called the Supplemental Nutrition Assistance Program 
(SNAP), but it excludes tax credits. Poverty estimates are based on comparisons with the official federal pov- 
erty levels for families of various sizes. The reward payments did not affect other public benefits that families 
may have been receiving, such as SNAP benefits, welfare payments under the Temporary Assistance for 
Needy Families (TANF) program, Medicaid, housing subsidies, and the EITC. 
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eat in the prior month) was 7.3 percentage points (or about 33 percent) lower than the control 
group average of 22.1 percent according to the 18-month survey (when all program group 
members were still in Family Rewards). 2 The magnitude of this effect was somewhat smaller by 
the time of the 42 -month survey (5.4 percentage points). Program group families were also less 
likely than those in the control group to report not having enough money to pay their rent some 
time in the year prior to the time they were interviewed. They were more likely to report having 
enough money to “make ends meet” and that their financial situation had improved over the 
prior year. Where these effects were evident in the early post-program period (as indicated by 
the 42 -month survey results), they were most likely reflecting a lingering effect of the cash 
transfers rather than a permanent change. 3 

• Family Rewards helped parents increase their savings and reduce their 
reliance on families and friends for cash loans. 

As Table 4.1 shows, the parents in Family Rewards were about 18 percentage points 
more likely than parents in the control group to report having a bank account after the program 
had ended. They were almost 8 percentage points more likely than those in the control group to 
have any savings. And they were about 5 percentage points less likely to have borrowed cash 
from family or friends in the prior year. 

• The reductions in hardships were large and heavily concentrated among 
families who were living in severe poverty at the time they began Family 
Rewards. 

Among families in the severe-poverty subgroup (that is, had an income that was less 
than 50 percent of the federal poverty threshold at the time of random assignment), those in 
Family Rewards were 14 percentage points less likely than similar families in the control group 
to have income below the federal poverty level in Year 3. (See Table 4.2.) Family Rewards also 
reduced this higher-poverty subgroup’s relative likelihood of experiencing food insufficiency 
after the program ended (according to the 42 -month survey) by 9 percentage points, and re- 
duced the relative likelihood of not paying their full rent in the past year by about 1 1 percentage 
points. 


Children’s Education 

Hie CCT programs instituted by lower- and middle-income countries have generally led to in- 
creases in school attendance. The effects have been larger at the secondary school level, since 


2 Slight discrepancies in percentages are a result of rounding. 

3 As discussed in Riccio et al. (2013a), post-program hardship reductions were smaller among survey re- 
spondents in the program group whose survey interviews took place later in the post-program period. 
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Table 4.2 

Impacts on Selected Outcomes Measuring Income, Poverty, and Material 
Hardship Through the Final Program Year or Early Post-Program Period, by 
Respondent's Poverty Level at the Time of Random Assignment 


Program 

Subgroup and Outcome Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Sig. 

Income at or above 50% of FPL at baseline 






Household income during Y ear 3 
(including Family Rewards payments) 

Average monthly income a,b,c ($) 

Annual income at or below FPL a b (%) 

2,093 

51.1 

1,771 

62.1 

323 *** 

-110 *** 

0.000 

0.000 


Household income during early post-program period 
(excluding Family Rewards payments) 

Annual income at or below FPL a b (%) 

62.4 

62.1 

0.2 

0.903 


Family "sometimes" or "often" did not have enough 
food to eat in past month (%) 

16.0 

19.0 

-3.0 

0.181 

f 

Family usually did not have enough money to make 
ends meet at end of month (%) 

34.7 

38.1 

-3.4 

0.227 


Family did not pay full rent or mortgage in 
past year d (%) 

42.9 

42.0 

0.9 

0.764 

ft 

Family's average savings exceed $50(P (%) 

13.7 

11.7 

2.0 

0.304 


Sample size (total = 1,193) 

642 

551 




Income less than 50% of FPL at baseline 






Household income during Y ear 3 
(including Family Rewards payments) 

Average monthly ineome a,b,c ($) 

Annual income at or below FPL a b (%) 

1,781 

63.5 

1,409 

77.2 

372 *** 

-13.6 *** 

0.000 

0.000 


Household income during early post-program period 
(excluding Family Rewards payments) 

Annual income at or below FPL a b (%) 

71.9 

77.2 

-5.3 * 

0.088 


Family "sometimes" or "often" did not have enough 
food to eat in past month (%) 

14.0 

23.2 

2 *** 

0.001 

t 

Family usually did not have enough money to make 
ends meet at end of month (%) 

36.4 

45.2 

-8.8 ** 

0.012 


Family did not pay frill rent or mortgage in 
past year d (%) 

35.7 

46.4 

-10.8 *** 

0.002 

ft 

Family's average savings exceed $500 e (%) 

10.3 

6.0 

4.2 ** 

0.035 


Sample size (total = 788) 

382 

406 
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Table 4.2 (continued) 


SOURCE: MDRC calculations using data from the Family Rewards 42-month survey. 

NOTES: Sample sizes may vary because of missing values. 

A two-tailed t-test was applied to differences between outcomes for the program and control 
groups. The p-value indicates the likelihood that the difference between the program and control groups 
arose by chance. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; 
* = 10 percent. 

Differences across subgroup impacts were tested for statistical significance. Statistical significance 
levels (Sig.) are indicated as follows: fff = 1 percent; f f = 5 percent; f = 10 percent. 

FPL is federal poverty level. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre -random 
assignment characteristics of sample members. 

Rounding may cause slight discrepancies in calculating sums and differences. 

See Riccio et al. (2010) and Riccio et al. (2013a, b) for more details on the measures and 
calculations used in this table. 

The items in the poverty, material hardship, and banking sections of the 42-month survey were 
administered to a random subsample of the survey respondents. 

The early post-program period refers approximately to the first six months since the time the 
program ended, which is when most respondents to the 42-month survey completed their interviews. 

a Monthly household income amounts equal to or greater than $10,000 were excluded from this 
calculation. 

b Annual household income is calculated by multiplying by 12 the respondent's income in the month 
prior to the survey interview. The federal poverty level was calculated based on annual income (monthly 
income multiplied by 12) and the household size at the time of the survey, applying the applicable 
poverty level threshold depending on when a respondent was interviewed. 

c The Year 1 income measures reported on the 18-month survey are within 3 percent of the 42- 
month income measures reported here. 

d Only about 4 percent of the 42-month survey sample owned an apartment or a house at the time of 
the survey. 

“About 6 percent of the 42-month survey sample is excluded from this analysis because of missing 
data. 


attendance at the primary (elementary school) level is often already relatively high. The effects 
have also been larger in areas or countries where attendance rates would have been lowest in the 
absence of the program. In some cases the education impacts were also larger among the poor- 
est households . 4 

However, few of these programs led to improvements in school performance. While 
none of them rewarded performance per se, the expectation was that students would leam more 
if they attended more often. Some researchers posited that the lack of effects was a result of 
poor schools and problematic home environments, neither of which was addressed by the 
CCTs. The exception to this finding is for very young children, where some studies suggest that 
cognitive gains were achieved for children who were of preschool age when their families par- 

4 Fiszbein and Schady (2009). 
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ticipated . 5 This finding is consistent with the increasingly popular idea that investments in early 
childhood can have important, lasting effects . 6 

In the United States and other higher-income countries, a growing number of studies 
have tested financial incentives linked to student performance. At the elementary and secondary 
school levels, for example, incentives have been offered to improve reading time, test perfor- 
mance, and course taking . 7 At the college level, some programs offer incentives linked to 
grades, courses taken, and persistence . 8 Studies that are particularly relevant to Family Rewards 
include tests of targeted incentives in three large U.S. school districts, including the SPARK 
program in New York City ; 9 a study of incentives for improving test performance in rural 
Ohio ; 10 a study of alternative fonns and timing of incentive offers in school districts in and near 
Chicago ; 1 1 and a study of incentives given to Israeli high school students to take and pass high 
school exit exams . 12 

A general finding from these studies is that incentives can increase student effort, at 
least in the short run. For example, the Israeli program got more students to take and pass re- 
quired high school exams, largely by encouraging some students to spend more time studying. 
However, it is not always clear that increased effort while incentives are offered will translate 
into higher achievement in the long run. For example, a program that paid students in rural Ohio 
for performance on annual standardized tests found positive effects on math scores during the 
program period but no effects in later years when the rewards were not offered. In contrast, a 
test of different incentives across three urban school districts found that they generally did not 
increase achievement, although there were positive effects for one subgroup in one city. In that 
study, incentives were offered to elementary school students for reading books (in Dallas); to 
high school students for good grades, with larger rewards for better grades (in Chicago); and to 
middle school students for proficient test performance (in the New York City SPARK pro- 
gram). The author acknowledges that each of the three tests did not have enough statistical 
power to detect notable effects, but concludes that, at best, the incentives had small to modest 
effects on performance . 13 The other Chicago study, which tested the timing of incentive pay- 
ments, found that incentive payments that are distributed quickly to students can be more effec- 

5 Paxson and Schady (2008). 

6 Carneiro and Heckman (2003). 

7 Bettinger (2010); Jackson (2010); Fryer (2011). 

8 Patel and Rudd (2012); Leuven, Oosterbeek, and van der Klaauw (2010); Angrist, Lang, and Oreopou- 
los (2009). 

9 Fryer (2011). As mentioned in Chapter 1, SPARK was a school-based education incentives program de- 
signed to improve the school performance of fourth- and seventh-graders by rewarding good performance on a 
series of standardized tests that were administered over the course of the academic year. 

10 Bettinger (2010). 

1 'Levitt, List, Neckermann, and Sadoff(2012). 

12 Angrist and Lavy (2009). 

13 Fryer (2011). 
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tive in improving test scores, and that the motivation to change behavior can vanish when incen- 
tive payments are delayed. 

Another finding from several of these studies is that the incentives had larger effects on 
students who were on the margins of higher performance, or students who were within reach of 
the outcome that was rewarded. For example, the positive effects on test scores for elementary 
school students in rural Ohio were larger for more academically prepared students. The program 
that offered rewards for passing high school exit exams in Israel similarly had larger effects for 
more prepared students. Put another way, these are the students who could have performed rela- 
tively well, but would not have attended as much or tried as hard without the incentive. 14 

More recently, research on CCTs in two Latin American countries has looked at longer- 
term effects on children’s school outcomes, with encouraging findings. For example, a non- 
experimental study of a Colombian CCT program found positive effects on high school gradua- 
tion for cohorts of students who were exposed to the program for anywhere from one to nine 
years. 15 A more powerful example comes from a program in Nicaragua. That study, based on an 
experimental design, found positive effects after 10 years on educational attainment and test 
scores for boys who were ages 9 to 12 when they entered the program. 16 The effects on 
achievement are notable, because they were measured seven years after the families of the 
young men stopped receiving the transfers. 

Effects on Education-Related Outcomes 

Coming from very-low-income families in six of New York’s highest-poverty neighbor- 
hoods, it is no surprise that the students in Family Rewards faced a range of challenges to their 
school progress. The magnitude of those challenges can be seen clearly from the experiences of 
the control group, who represent the “norm,” or students’ experience in the absence of Family 
Rewards. 17 For example, only 86 percent of the children in the control group who began the 
study in fourth grade had progressed to ninth grade (the expected grade level) by Year 6. Chil- 
dren in the control group who were entering seventh grade at the time of random assignment 
were even less likely to keep up: by Year 6, only 43 percent of the students in that seventh-grade 
cohort had graduated on time, although 8 1 percent were still officially enrolled in school. 

Among entering ninth-graders, the percentage who were on track plummeted over the 
ensuing years. The falloff was greatest between the ninth and tenth grades (Years 1 and 2 of the 


14 Slavin (2010). In his review of incentives for school progress, Slavin questions the cost-effectiveness of 
these programs, arguing that other types of reforms, such as those that change an entire school, might affect 
more students than just those on the margin of success. 

15 Baez and Camacho (2011). 

16 Barham, Macours, and Maluccio (2013). 

17 See Miller and Deitch (2016), available at www.mdrc.org, for these data. 
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study). By Year 4, only 51 percent of control group members who were entering ninth grade at 
the start of the study were enrolled in twelfth grade (as shown in Table 4.3). Attendance also fell 
with each passing year. Among those in the control group who were enrolled in Year 4, for ex- 
ample, the average attendance rate was only 60 percent. Finally, only 48 percent of the ninth- 
grade control group cohort had graduated by the end of Year 4. The graduation rate improved 
only marginally by Year 6, rising only to 52 percent. 

It was against this backdrop that Family Rewards sought to make a difference in chil- 
dren’s school perfonnance. 

• Family Rewards did not improve school outcomes for elementary or 
middle school students. 

For elementary and middle school students, the analysis found few positive effects on at- 
tendance rates, scores on standardized tests, or other school outcomes during the program period 
or by the end of Year 6 (not shown in the tables). 18 In addition, subgroup analyses did not reveal 
any consistent patterns of positive effects for particular types of students in those grades. Perhaps 
the limited approach of the Family Rewards model for these children — that is, rewarding only 
attendance and standardized test scores above a certain level (rather than more immediate per- 
fonnance indicators, such as good report card grades), and paying all rewards to the parents — 
might explain in part why the program did not have an educational payoff for this group. 

• Family Rewards had few effects on school outcomes for high school stu- 
dents overall. However, it substantially improved school performance 
and increased graduation rates for students who were already stronger 
readers. 

Generally speaking, as Table 4.3 shows, ninth-grade students in Family Rewards over- 
all perfonned no better on school outcomes than did their control group counterparts. Both 
groups had disappointing outcomes, with just under half graduating within four years, and just 
over half graduating within six years. 

The picture changes when the results are examined separately according to students’ 
academic readiness for high school, according to their scores on their eighth-grade standardized 
tests. Those who entered better-prepared for high school — who may have been in a better po- 
sition to take advantage of the incentives offer — do appear to have benefited from Family Re- 
wards. In contrast, ninth-grade students who were behind educationally when they entered Fam- 
ily Rewards did not experience educational gains from the program. It may be that, on average, 


18 See Miller and Deitch (2016), available at www.mdrc.org, for these data. 
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Table 4.3 


Impacts on Selected Education Outcomes for Students 
in Grade 9 at Random Assignment 



Program 

Control 

Difference 


Outcome 

Group 

Group 

(Impact) 

P-Value 

Enrollment and graduation (%) 

Enrolled in grade 12 in Year 4 

53.1 

51.2 

1.9 

0.360 

Enrolled in any grade in Y ear 4 

80.1 

79.2 

0.9 

0.610 

Enrolled in any grade in Y ear 6 

8.8 

10.8 

-2.1 

0.123 

Graduated within 4 years 

49.2 

48.2 

1.1 

0.621 

Graduated within 6 years 

53.9 

52.3 

1.6 

0.420 

Dropped out within 6 years 

18.1 

19.3 

-1.1 

0.505 

Attendance rate 95% or higher (%) 

Year 1 

34.0 

31.5 

2.5 

0.211 

Year 2 

28.8 

23.7 

^ i *** 

0.007 

Year 3 

25.1 

21.9 

3.1 * 

0.089 

Year 4 

17.4 

15.3 

2.1 

0.197 

Average attendance rate (%) 

Year 1 

81.8 

81.4 

0.4 

0.683 

Year 2 

75.3 

74.3 

1.0 

0.439 

Year 3 

69.4 

67.7 

1.7 

0.254 

Year 4 

60.7 

59.7 

1.1 

0.508 

Credits earned 

Average number of credits earned, Years 1 to 4 

32.7 

31.9 

0.8 

0.300 

Average number of credits earned. Years 1 to 6 

33.7 

33.4 

0.4 

0.667 

Earned at least 44 credits, Y ears 1 to 4 (%) 

41.5 

40.5 

0.9 

0.652 

Earned at least 44 credits, Y ears 1 to 6 (%) 

45.5 

45.7 

-0.2 

0.923 

Regents exams passed (%) 

Passed at least 5 Regents exams, Years 1 to 4 

36.7 

35.7 

1.1 

0.562 

Passed at least 5 Regents exams, Years 1 to 6 

38.1 

38.2 

-0.1 

0.942 

Rewards earned. Years 1 to 3 a ($) 

Total amount earned for education activities 

2,868 




Attendance 

534 

— 

- 

- 

Regents exams 

1,370 

- 

- 

- 

Earning 11+ credits 

855 

— 

— 

— 

Sample size (total = 1,978) 

988 

990 




SOURCE: MDRC calculations using data from New York City Department of Education 
administrative records and Seedco's Family Rewards program data. 


NOTES: Sample sizes may vary because of missing values. 

A two-tailed t-test was applied to the differences between outcomes for the program and control 
groups. The p-value indicates the likelihood that the difference between the program and control 
groups arose by chance. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 
5 percent; * = 10 percent. 
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Table 4.3 (continued) 


Estimates were regression-adjusted using ordinary least squares, controlling for pre -random assign- 
ment characteristics of sample members. Standard errors were adjusted to account for multiple observa- 
tions per family. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Years 1, 2, 3, 4, 5, and 6 cover the 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, and 
2012-2013 school years, respectively. 

The Regents measures in this table include the following Regents exams: English, Math A, Math B, 
Geometry, Integrated Algebra, Algebra 2/Trigonometry, U.S. Elistory and Government, Global History 
and Geography, Living Environment, Chemistry, Physics, and Earth Science. 

A double dash (— ) indicates "not applicable." 

a Reward amounts are calculated across all program group members, including those who did not earn 
any education rewards. The total amount includes other rewards for education activities not listed. 


this group was just too far behind educationally or too disengaged from school for the incen- 
tives to make much of a difference. Although subgroup findings tend to carry less statistical cer- 
tainty than full-sample estimates, these results are consistent with some prior research: as previ- 
ously mentioned, a number of studies of education-focused incentives programs found more 
positive effects for more capable students. 19 

Family Rewards had particularly strong effects on students in the ninth-grade cohort 
who had scored at or above the basic proficiency level on their eighth-grade standardized Eng- 
lish language arts (ELA) test (which primarily tests reading skills) before random assignment. 20 
For this subgroup, which made up almost one-third of the overall sample of ninth-graders, Fam- 
ily Rewards appears to have improved a range of school outcomes. (See Table 4.4.) These im- 
provements include positive impacts on, among other outcomes, the proportion of ELA- 
proficient students who were enrolled in grade 12 in Year 4 (and thus headed toward on-time 
graduation), on their likelihood of earning at least 44 credits (the amount needed to graduate), 
and on their likelihood of passing at least five New York State Regents exams. 21 It also pro- 
duced an 8 percentage point increase in the likelihood of on-time graduation from high school 


19 See, for example, Angrist and Lavy (2009). 

20 In 2010, amid concerns that these tests had become too easy and not reflective of the proficiency needed 
in each grade, the New York State Department of Education raised the scores necessary to be deemed profi- 
cient. In 2009 and earlier, a score of 650 or higher was required in order to be deemed proficient. Starting in 
2010, the cutoff scores were raised to a range of 658 to 684, depending on the grade level and the test. As a 
result, proficiency rates citywide fell dramatically between 2009 and 2010, as they did for students in the Fami- 
ly Rewards study. 

2 'Students must pass at least five tests in specified subject areas in order to graduate with a diploma recog- 
nized by the New York State Board of Regents, which sets standards and regulations for all public schools in 
the state. 
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Table 4.4 


Impacts on Selected Education Outcomes for Students in Grade 9 at Random Assignment, 
by Performance on English Language Arts (ELA) Test in the Prior Year (Grade 8) 



Proficient on 

8th Grade ELA Test 3 

Not Proficient on 

8th Grade ELA Test 3 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Sig. 

Enrollment and graduation (%) 

Enrolled in grade 12 in Year 4 

78.4 

68.2 

IQ | *** 

48.3 

50.3 

- 2.0 

tt 

Enrolled in any grade in Year 4 

90.9 

89.0 

1.9 

82.7 

81.3 

1.4 


Enrolled in any grade in Year 6 

4.4 

5.9 

- 1.6 

9.9 

12.9 

-3.0 


Graduated within 4 years 

74.8 

66.9 

8.0 ** 

43.2 

45.9 

- 2.8 

tt 

Graduated within 6 years 

81.7 

72.2 

g 5 *** 

50.8 

52.5 

- 1.6 

tt 

Attendance rate 95% or higher (%) 

Year 1 

54.9 

42.3 

12.6 *** 

29.6 

29.6 

0.1 

tt 

Year 2 

47.0 

33.9 

23 2 *** 

24.3 

21.5 

2.8 

tt 

Year 3 

40.3 

30.8 

9.5 ** 

21.4 

20.4 

1.0 

t 

Year 4 

28.1 

23.0 

5.2 

14.8 

14.3 

0.5 


Average attendance rate (%) 

Year 1 

92.0 

86.9 

5 j *** 

82.1 

82.3 

- 0.2 

ttt 

Year 2 

87.5 

81.4 

6 i *** 

74.9 

75.1 

-0.3 

tt 

Year 3 

83.9 

77.1 

6 8 *** 

68.1 

67.7 

0.4 

tt 

Year 4 

76.7 

71.6 

5.1 * 

59.8 

60.4 

- 0.6 


Credits earned 

Average number of credits earned. Years 1 to 4 

44.3 

40.0 

4 3 *** 

30.9 

31.8 

- 0.8 

ttt 

Average number of credits earned, Years 1 to 6 

45.2 

41.2 

3 g *** 

32.1 

33.6 

-1.4 

ttt 

Earned at least 44 credits, Years 1 to 4 (%) 

66.1 

56.6 

9.6 ** 

36.9 

39.8 

- 2.8 

tt 

Earned at least 44 credits, Years 1 to 6 (%) 

71.3 

60.8 

10.5 ** 

41.0 

46.3 

-5.3 * 

ttt 

Regents exams passsed (%) 

Passed at least 5 Regents exams, Years 1 to 4 

72.5 

63.1 

9.5 ** 

25.8 

28.9 

-3.1 

ttt 

Passed at least 5 Regents exams, Years 1 to 6 

74.0 

65.4 

8.6 ** 

27.1 

31.4 

-4.3 * 

ttt 
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Table 4.4 (continued) 



Proficient on 

8th Grade ELA Test a 

Not Proficient on 

8th Grade ELA Test a 


Outcome 

Program 

Group 

Control Difference 
Group (Impact) 

Program 

Group 

Control Difference 
Group (Impact) 

Sig. 

Rewards earned. Years 1 to 3 b ($) 

Total amount earned for education activities 
Amount earned from attendance 

Amount earned from Regents exams 

Amount earned from earning 11+ credits 

4,536 

800 

2,276 

1,308 

- 

2,447 

483 

1,109 

764 

- 


Sample size (total = 1,700) 

271 

256 

576 

597 



SOURCE: MDRC calculations using data from New York City Department of Education administrative records and Seedco's Family Rewards 
program data. 

NOTES: Sample sizes may vary because of missing values. 

A two-tailed t-test was applied to the differences between outcomes for the program and control groups. The p-value indicates the 
likelihood that the difference between the program and control groups arose by chance. Statistical significance levels are indicated as follows: 
*** = i percent; ** = 5 percent; * = 10 percent. 

Differences across subgroup impacts were tested for statistical significance. Statistical significance levels (Sig.) are indicated as follows: 
ttt = 1 percent; ft = 5 percent; f = 10 percent. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample 
members. Standard errors were adjusted to account for multiple observations per family. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Years 1, 2, 3, 4, 5, and 6 cover the 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, and 2012-2013 school years, respectively. 

The Regents measures in this table include the following Regents exams: English, Math A, Math B, Geometry, Integrated Algebra, 
Algebra 2/Trigonometry, U.S. History and Government, Global History and Geography, Living Environment, Chemistry, Physics, and Earth 
Science. 

A double dash (--) indicates "not applicable." 

a In New York State, students who score at a level of 3 or higher on a 4-point scale are deemed "proficient." 

b Reward amounts are calculated among all program group members in each subgroup, including those who did not earn any education 
rewards. The total amount includes other rewards for education activities not listed. 



within four years (a gain of 12 percent above the 67 percent graduation rate among control 
group students who were ELA-proficient at the beginning of the study). The graduation-rate 
impact rose to 10 percentage points (or 13 percent) by the end of Year 6. 22 These effects are par- 
ticularly noteworthy because they occurred without any changes in the schools themselves or in 
teachers’ instructional practices. The positive effects also tended to be larger and more consist- 
ently statistically significant among girls than among boys in the proficient-reader subgroup 
(although possibly owing to small subgroup sample sizes, the differences in impacts by gender 
are not themselves statistically significant). 23 

For the ninth-graders who were proficient on their eighth-grade math test, Family Re- 
wards produced positive effects on various educational outcomes during the program phase on- 
ly (not shown in tables). 24 For example, it improved their attendance rates and credit accumula- 
tion while they were in the program. However, these positive effects did not persist into Year 4, 
when the incentives were no longer available. In addition, the math-proficient subgroup did not 
experience an increase in graduation rates. 

Why Family Rewards would have stronger and more sustained effects for reading- 
proficient students is not immediately clear. It was not because math-proficient students were 
less capable students overall. In fact, among control group members, those who were proficient 
in math were somewhat more likely than those who were proficient in reading to graduate with- 
in four years (7 1 percent versus 67 percent). 25 In the absence of the program, perhaps reading- 
proficient students, on average, would have been more likely than math-proficient students to 
perform below their potential (possibly in math courses, in particular), leaving them more room 
to improve by investing more effort, as inspired by the incentives. 

• Family Rewards led high school students to change how they spent their 
time when they weren’t in school, which may have contributed to the 
educational gains that the more proficient students experienced. 

The findings from a special study that was conducted as part of the evaluation showed 
that high school students in the program group shifted more of their attention during nonschool 


22 As Table 4.4 shows, on most outcome measures, the impacts for ELA-proficient students were statisti- 
cally significantly different from the impacts for students who were not ELA-proficient. 

“ These results are presented in Appendix Table G.16 in Riccio et al. (2013b), which is available at 
www.mdrc.org. 

24 See Miller and Deitch (2016), available at www.mdrc.org. 

25 See Tables S2.4 and S2.5 in Miller and Deitch (2016), available at www.mdrc.org. It is also noteworthy 
that when they entered the study, ninth-graders who were proficient in reading were not necessarily proficient 
in math, and vice versa. On the students’ eighth-grade standardized tests, the correlation between ELA profi- 
ciency and math proficiency was about 0.44. 
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Box 4.2 


Using Incentives to Change the Way Teenagers Spend Their Time 

The Family Rewards evaluation included a special study of high school students to 
discover whether the program affected the way they spent their time when they were 
not in school. It also examined effects on other outcomes, such as their mental health, 
aggressive behavior, substance use, and family spending patterns. The study found 
that Family Rewards: 

• Changed how teenagers spent their time when not in school, increasing the propor- 
tion who engaged primarily in academic activities and reducing the proportion 
who engaged primarily in social activities relative to similar teenagers in the con- 
trol group. 

• Had the most pronounced effect on use of time among the teenagers who were 
better prepared for high school (that is, had scored at proficiency levels on their 
eighth-grade standardized math test when they entered Family Rewards. 

• Increased parents’ spending on school-related and leisure activities and supplies, 
and increased the proportion of parents who saved money for their children’s future 
education. 

• Had no effects on parents’ monitoring of their teenage children’s activities or be- 
havior and did not increase parent-teenager conflict (as might have been expected by 
paying substantial cash rewards directly to teenagers) or the teenagers’ depression or 
anxiety. 

• Did not reduce teenagers’ intrinsic motivation by paying them rewards for 
school attendance and academic achievement. 

• Substantially reduced teenagers’ self-reported problem behavior, such as aggression 
and substance use. 

(continued) 


hours from social activities to academic activities (for example, doing homework and partici- 
pating in achievement-oriented after-school activities), compared with the way students in the 
control group spent their time when they were not in school. 26 (See Box 4.2.) This change was 
concentrated among students in the more proficient subgroup. As such, it may be one of the 
mechanisms through which the program improved the school outcomes for that subgroup. 

26 Morris, Aber, Wolf, and Berg (2012). 
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• The effects of Family Rewards were no better or worse for students who 
were enrolled in higher-performing schools or in lower-performing 
schools or for students with more versus less educated parents. 

As part of the analysis of effects on school outcomes, the evaluation prespecified three 
dimensions across which the program’s effects might vary: students’ past performance, parents’ 
education level, and students’ school enviromnent. The results showed little variation in effects 
across these subgroups for the elementary and middle school students. For entering ninth- 
graders, unlike the effects by students’ past perfomiance, the effects of the program did not dif- 
fer significantly by parents’ education level or by school enviromnent. 
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School environment — that is, whether a school was “low-performing” or “high- 
performing” — was defined using test scores of earlier cohorts in the school a student entered 
when he or she entered the study. Specifically, students’ schools were ranked according to their 
average pass rates for the English and math Regents exams in the 2005-2006 and 2006-2007 
school years. The schools were then divided into thirds (low-, medium-, and high-performing) 
based on this ranking. The findings indicate that school environment is strongly associated with 
outcomes — for example, students who entered the study in the lowest-ranked schools had low- 
er attendance rates and graduation rates than did their counterparts who attended higher-ranking 
schools. Nonetheless, the program’s impacts were not found to differ between higher- and low- 
er-performing schools. 27 

• The education incentives do not appear to have reduced students’ in- 
trinsic motivation. 

As discussed in Chapter 1, some critics of incentives believe that extrinsic rewards 
might reduce children’s intrinsic motivation to leam, especially after the incentives end, harm- 
ing their educational outcomes. There is no indication that Family Rewards caused any con- 
sistent pattern of statistically significant negative effects on school outcomes, even after the 
program ended. For example, for the group for whom the program had the largest effect — 
reading-proficient ninth-graders — the behavioral changes in terms of attendance and credits 
attempted lessened in general once the incentives were removed. But students in the program 
group did not perform systematically worse on these or other outcomes than did those in the 
control group. 28 Moreover, the special study of ninth-graders measured intrinsic motivation di- 
rectly by asking study participants a series of questions (similar to those used in other studies) 
about engagement in school and motivation to leam. That analysis found no difference in intrin- 
sic motivation between the program and control group students. 29 


27 Wolf (2014) examined how the program’s impacts varied by school environment, using a more compre- 
hensive definition of school environment that included attendance rates, per-pupil expenditures, and students’ 
perceptions of school safety as well as test scores. For a subset of ninth-graders, she found that effects on aca- 
demic achievement in Y ear 3 were positive at lower-quality schools and negative at higher-quality schools, for 
an inverse relationship. However, this finding was not evident in an analysis that was conducted for the current 
report for the full sample of ninth-graders, or for other measures of achievement, such as four-year and six-year 
graduation rates. The discrepancy may have to do with differences in the composition of the samples across the 
two analyses. See, also, Riccio et al. (2013a) for further details on an earlier full-sample analysis. 

~ 8 Among the nonproficient readers, Table 4.4 does show some outcomes for the program group that are 
lower than those for the control group. However, most of the differences are small and not statistically signifi- 
cant, and thus do not suggest a meaningful pattern of negative effects. 

29 Morris, Aber, Wolf, and Berg (2012). Similarly, Fryer (2011) found little evidence that the school-based 
incentives programs that he studied caused reductions in intrinsic motivation. 
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• Family Rewards reduced teenagers’ problem behaviors, key outcomes 
that the intervention did not target directly. No increases were found in 
teenagers’ depression or anxiety. 

The special study of teenagers’ use of time also examined the possible effects of Family 
Rewards on teenagers’ mental health and problem behaviors. 30 It found that teenagers in the 
program group reported lower levels of aggression and lower levels of alcohol and marijuana 
use, and a lower likelihood of having friends using those substances, compared with their peers 
in the control group. It may be that the shifts in use of time from more social to more academic 
activities resulted in less opportunity to engage in problem behaviors and possibly a change in 
time spent with peers who engage in such behaviors. The program also had no statistically sig- 
nificant impacts on teenagers’ depression and anxiety. Given concerns that parents would put 
undue pressure on their children and increase their anxiety levels as a result, this “null” finding 
should be seen as positive. 

• Family Rewards increased parents’ spending on their teenagers and 
savings for their teenagers’ future education, indicating positive effects 
on parents’ investment in children’s human capital. 

By providing cash transfers to families, Family Rewards should have resulted in greater 
spending and, perhaps, savings within families. However, whether or not those increased re- 
sources would be spent in ways that would benefit children’s long-term human capital was an 
open question because the program placed no constraints on how families could spend their re- 
ward payments. The special study of teenagers’ use of time examined this question and found 
that parents in the program group had higher levels of monthly spending on academic activities 
and supplies and on leisure and entertainment than did parents in the control group. 31 Notably, 
Family Rewards also had a positive impact on parents’ indicating that they were currently sav- 
ing for their teenagers’ future education. 

• Family Rewards did not increase the rate of enrollment in postsecond- 
ary educational programs, but it does seem to have encouraged reading- 
proficient ninth-graders to enroll full time, rather than part time, and to 
enroll in four-year colleges rather than two-year institutions. 

Following students for six years after random assigmnent, the evaluation can track 
many of them into college or postsecondary technical institutes. This tracking is possible using 
data from the National Student Clearinghouse, which collects data on postsecondary enrollment 
and degree receipt nationwide, covering 95 percent of postsecondary enrollment in the United 


30 Morris, Aber, Wolf, and Berg (2012). 
31 Morris, Aber, Wolf and Berg (2012). 
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States. 32 For the Family Rewards ninth-grade cohort, the Clearinghouse data cover two years 
after high school graduation for those who graduated within four years. 

As Table 4.5 shows, 42.5 percent of ninth-graders in the control group enrolled in a 
postsecondary institution at some point during the six-year follow-up period. Family Rewards 
had no statistically significant effect on this outcome. A more relevant analysis may be for the 
reading-proficient ninth-graders (shown in Table 4.6), since the program increased high 
school graduation rates for this group, better positioning them for enrollment in college or a 
technical institute. Although 62.5 percent of the reading-proficient program subgroup enrolled 
in such an institute (according to National Clearinghouse data), this rate was about the same 
as the rate for reading-proficient students in the control group. However, it does seem that 
Family Rewards caused the proficient students to move away from two-year institutions and 
part-time enrollment toward full-time enrollment in four-year colleges. For example, the pro- 
gram increased full-time enrollment in four-year colleges by 9.3 percentage points for the 
ELA-proficient subgroup. 33 

• Family Rewards may have improved early school outcomes for younger 
children, who were ages 2 to 7 when their families enrolled in the study. 

A growing literature suggests that increases in family income can have positive effects 
on children’s well-being and school progress, particularly for very young children. 34 As it turns 
out, a substantial number of families in the Family Rewards evaluation had very young chil- 
dren, who ranged from preschoolers to kindergarteners when their parents enrolled in the study. 
These children were eligible for the health rewards but not the education rewards, unless they 
reached elementary school during the three-year program period. 35 However, they would have 
experienced increased family incomes during their preschool years or very early elementary 
school years, primarily from the rewards that their parents and older siblings received. By the 
fifth and sixth years of follow-up, many of these children were old enough to have taken stand- 
ardized tests, which are administered starting in third grade. The evaluation thus had an oppor- 
tunity to examine whether these children in the program group, who lived in households with 


32 Dynarski, Hemelt, and Hyman (2015). The authors found very high rates of coverage for public institu- 
tions and lower rates for for-profit institutions. 

33 The reasons why students in the Family Rewards group who were not proficient in reading, compared 
with those in the control group, may have been less likely to enroll in any college or to enroll full time 
are uncertain. For example, it is not clear whether they were more discouraged about further schooling, or more 
likely to be working, or more likely to be enrolled in vocational training or General Educational Development 
(GED) programs not captured by the National Clearinghouse data. It may be an issue worthy of further investi- 
gation. 

34 See, for example, Dahl and Lochner (2012); Magnuson et al. (2013); Duncan, Magnuson, and Votruba- 
Drzal (2014); and Miller et al. (2008). 

35 Attendance rewards began in first grade and test score rewards began in third grade. 
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Table 4.5 


Impacts on Postsecondary Enrollment for Students in Grade 9 at the Time of 

Random Assignment 


Grade Level and Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Ever within 6 years after baseline (%) 

Enrolled at any postsecondary institution 

39.9 

42.5 

-2.6 

0.204 

2 -year 

20.5 

22.5 

-2.0 

0.281 

4 -year 

22.5 

22.8 

-0.4 

0.841 

Highest level ever enrolled was full time 

27.5 

28.8 

-1.3 

0.510 

Highest level ever enrolled was part time 

3.9 

3.9 

0.0 

0.993 

Enrolled in 4-year and full time 

18.8 

17.7 

1.2 

0.475 

Enrolled in 2-year and full time 

11.2 

13.5 

-2.3 

0.127 

Sample size (total = 1,978) 

988 

990 




SOURCE: MDRC calculations using data from the National Student Clearinghouse. 

NOTES: Sample sizes may vary because of missing values. 

A two-tailed t-test was applied to the differences between outcomes for the program and 
control groups. The p-value indicates the likelihood that the difference between the program and 
control groups arose by chance. Statistical significance levels are indicated as follows: *** = 1 
percent; ** = 5 percent; * = 10 percent. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 
assignment characteristics of sample members. Standard errors were adjusted to account for 
multiple observations per family. 

Rounding may cause slight discrepancies in calculating sums and differences. 


higher incomes than those in the control group during the period the program operated (because 
of the reward payments), had any better school outcomes. 

Table 4.7 shows the results for these children on the standardized tests they took in the 
evaluation’s sixth year of follow-up, when they were in the third, fourth, or fifth grade (depend- 
ing on their ages). As can be seen from this exploratory analysis, Family Rewards increased the 
proportion of these children who scored at proficiency levels on the ELA test in Year 6 by 5.3 
percentage points and on the math test by 5.5 percentage points. These impacts are consistent 
with other experiments showing the positive effects that increases in family income (resulting 
from participation in welfare-to-work and related programs that included earnings supplements) 
can have on young children’s school performance. 36 However, given the attention that Family 


36 Morris et al. (2001). 
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Table 4.6 


Impacts on Postsecondary Enrollment for Students in Grade 9 at the Time of Random Assignment, by 
Performance on English Language Arts (EL A) Test in the Prior Year (Grade 8) 

Proficient on Not Proficient on 

8th Grade ELA Test* 1 8th Grade ELA Test J 


Subgroup and Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Sig. 

Ever within 6 vears after baseline (%) 

Enrolled at any postsecondary institution 

62.5 

61.2 

1.4 

33.8 

38.9 

-5.0 * 


2-year 

21.3 

26.3 

-5.0 

21.5 

23.1 

-1.6 


4-year 

46.0 

38.8 

7.1 * 

15.4 

18.2 

-2.8 

tt 

Highest level ever enrolled was full time 

49.9 

40.6 

q 3 ** 

21.4 

26.8 

-5.4 ** 

ttt 

Highest level ever enrolled was part time 

4.7 

4.9 

-0.2 

4.2 

4.0 

0.2 


Enrolled in 4-year and full time 

40.9 

31.2 


12.2 

13.5 

-1.4 

tt 

Enrolled in 2-year and full time 

12.4 

13.7 

-1.3 

11.9 

14.8 

-2.9 


Sample size (total = 1,700) 

271 

256 


576 

597 




SOURCE: MDRC calculations using data from the National Student Clearinghouse. 

NOTES: Sample sizes may vary because of missing values. 

A two-tailed t-test was applied to the differences between outcomes for the program and control groups. The p-value indicates the 
likelihood that the difference between the program and control groups arose by chance. Statistical significance levels are indicated as follows: 
*** = i percent; ** = 5 percent; * = 10 percent. 

Differences across subgroup impacts were tested for statistical significance. Statistical significance levels (Sig.) are indicated as follows: 
ttt = 1 percent; ft = 5 percent; f = 10 percent 

Estimates were regression-adjusted using ordinary least squares, controlling for pre -random assignment characteristics of sample members. 
Standard errors were adjusted to account for multiple observations per family. 

Rounding may cause slight discrepancies in calculating sums and differences. 

a In New York State, students who score at a level of 3 or higher on a 4-point scale are deemed "proficient." 



Table 4.7 


Impacts on Test Scores for Children Ages 2 to 7 at the Time of Random 

Assignment 


Grade Level and Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Year 6, Grades 3-5 





ELA scale score 

288.7 

287.6 

1.2 

0.611 

Scored at proficient level or higher on ELA a 

18.1 

12.8 

5.3 ** 

0.048 

Math scale score 

288.6 

285.8 

2.8 

0.227 

Scored at proficient level or higher on math a 

21.4 

15.9 

5.5 * 

0.062 

Average attendance rate (%) 

85.8 

85.1 

0.7 

0.686 

Attendance rate 95% or higher (%) 

39.2 

39.2 

0.0 

0.996 

Sample size (total = 788) 

410 

378 




SOURCE: MDRC calculations using data from New York City Department of Education 
administrative records. 

NOTES: Sample sizes may vary because of missing values. The ages of 2 to 7 are based on the age 
recorded in the Baseline Information Form. There are three children included in this sample whose 
ages fall outside this range. 

A two-tailed t-test was applied to the differences between outcomes for the program and control 
groups. The p-value indicates the likelihood that the difference between the program and control 
groups arose by chance. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 
5 percent; * = 10 percent. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre -random 
assignment characteristics of sample members. Standard errors were adjusted to account for multiple 
observations per family. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Year 6 covers the 2012-2013 school year. 

Proficiency in Y ear 6 is based on the Common Core State Standards. 

a ln New York State, students who score at a level of 3 or higher on a 4-point scale are deemed 
"proficient." 


Rewards focused on children’s education in general, the results may have also been driven in 
part by changes that might have occurred in parents’ attitudes toward or support for their young 
children’s education (although these indicators were not measured). 


Health Care Behaviors and Health Conditions 

International CCT programs have been found to increase the use of health services . 37 For exam- 
ple, a number of studies have shown impacts on visits to health clinics, especially for very 

37 Fiszbein and Schady (2009); Lagarde, Elaines, and Palmer (2007). 
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young children, who are often the focus of health care incentives. Evidence on health outcomes, 
however, is more mixed. Some studies find increased height among the youngest children, 
while others find no effects on health status. 

In terms of more targeted programs in the United States, a growing body of research 
documents that incentives can affect a variety of health-related behaviors, such as weight loss , 38 
smoking , 39 and adherence to prescribed medication . 40 In addition, a recent summary of studies 
found that the majority of incentives programs that were reviewed — providing rewards in the 
form of cash, lotteries, gifts, or coupons — were found to affect individuals’ behavior . 41 Anoth- 
er review suggests that the payments need not be large to affect a variety of health-related out- 
comes . 42 However, most of these studies were small, clinical trials, and most of them evaluated 
behavioral change in the short term only. In the few studies that did track long-term outcomes, 
the effects typically faded after the intervention ended. 

Most relevant to Family Rewards are incentives for more general preventive practices, 
such as maintaining health insurance and visiting the doctor and dentist. Evidence on the effects 
of incentives to maintain health insurance is scant, but recent research shows that having health 
insurance does increase the use of preventive care . 43 

Effects on Health-Related Outcomes 

• Family Rewards did not increase families’ use of preventive medical 
care, which was already high, and it had few effects on health outcomes. 

The health-related incentives of the Family Rewards program were designed to encour- 
age low-income families to adopt better preventive health care practices. It turned out that a 
higher proportion of families than the program’s designers had expected were already receiving 
health insurance coverage and practicing preventive health care . 44 The high rate in New York 
City, which was higher than in many other cities, likely reflects efforts by the city and state to 
improve access to health care coverage and to improve the health care delivery system for low- 
and moderate-income families . 45 Most parents in the control group reported having public 


38 Volpp et al. (2008a). 

39 Volpp et al. (2009) and Volpp et al. (2015). 

40 Volpp et al. (2008b). 

41 Kane, Johnson, Town, and Butler (2004). 

42 Sindelar (2008). 

43 Finkelstein et al. (2012). 

44 The study sample did not include low-income single adults or undocumented immigrants, who are much 
less likely to have health insurance. 

45 See United Hospital Fund (2009) for a list of reforms that New York State implemented in 2007. That 
report documents that in the year after the reforms were introduced, 85 percent of eligible children were cov- 
ered by Medicaid or the State Children’s Health Insurance Program, a rate substantially above the national 
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health insurance (Medicaid and the State Children’s Health Insurance Program) for their fami- 
lies, reported getting regular physicals for themselves and their children, and said they had a 
regular place for receiving routine health services (a “medical home”). Consequently, on some 
measures, Family Rewards had much less room to improve the use of basic preventive health 
care services further. 

Perhaps for that reason, Family Rewards had few noteworthy health-related impacts, 
according to the 42 -month survey. (See Table 4.8.) It did reduce the likelihood that parents or 
their children would experience an interruption in health insurance coverage in the prior year by 
over 2 percentage points (even after the health insurance rewards were discontinued in the third 
program year). But it did not improve the likelihood that parents or children would get health 
checkups or see health professionals for other reasons, primarily because most families received 
those services already. A small reduction in families’ use of emergency rooms for routine medi- 
cal care that was evident from the evaluation’s 18-month survey faded, with very few families 
in either the program or control group reporting on the 42 -month survey that they had used 
emergency rooms for that purpose. 

• Family Rewards produced large increases in fam il ies'’ use of dental care 
services. 

Family Rewards led to increased dental care for parents and children alike. For exam- 
ple, according to responses to the 42 -month survey, parents in the program group were 10 per- 
centage points more likely than control group parents to report having seen a dentist for any rea- 
son in the prior year, and about 12 percentage points more likely to have had two dental check- 
ups or more in the past year. Strong positive effects were also observed among high school stu- 
dents (for example, a 19 percentage point increase in two dental checkups in the past year) and 
among younger children (not shown). The importance of oral health has been well documented, 
in addition to disparities by income level in access to oral health services. 46 Poor oral health can 
affect children in the short run, but it can also lead to a number of more serious medical condi- 
tions as they age. 47 

• Family Rewards may have improved health outcomes among those who 
had significant health problems when they entered the program. 


average. Similarly, data for 2009 from the U.S. Census Bureau indicate that the percentage of low-income 
children without insurance was lower in New York State than in the United States, and the rate of government- 
provided coverage for this group was higher in New York State than in the United States on average. 

46 Fisher-Owens et al. (2008). 

47 A recent study also documents effects of poor oral health on labor market outcomes (Glied and Neidell, 
2010 ). 
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Table 4.8 


Impacts on Selected Outcomes Measuring Health Care 
Through the Final Program Year or Early Post-Program Period 



Program 

Control 

Difference 


Outcome 

Group 

Group 

(Impact) 

P-Value 

Parents' use of health insurance (%) 





Had a period with no health insurance 
coverage in past 12 months 

15.3 

17.6 

-2.3 * 

0.087 

Sample size (total = 2,966) 

1,543 

1,423 



Parents' use of health services and health status 





Uses hospital emergency room as usual source 
of care when sick 

3.0 

4.0 

- 1.0 

0.238 

Has seen health professional for any reason 
in past 12 months 

94.4 

94.5 

- 0.1 

0.924 

Used emergency room for own health problem 
in past 12 months 

44.2 

46.2 

- 2.0 

0.378 

Had a health checkup in past 12 months 

90.0 

88.9 

1.1 

0.427 

Currently has, and is being treated for, 
any medical condition 3 

50.1 

47.8 

2.3 

0.276 

Asthma 

15.0 

14.9 

0.1 

0.942 

High blood pressure/hypertension 

25.6 

24.8 

0.8 

0.672 

High cholesterol/high LDL 

9.0 

9.4 

-0.5 

0.709 

Diabetes 

11.2 

10.4 

0.8 

0.568 

Smokes cigarettes 

22.6 

24.5 

-1.9 

0.314 

Self-rated health is "excellent" or "very good" 

35.9 

32.9 

3.0 

0.137 

Has seen dentist for any reason in past 12 months 

85.4 

75.3 

i *** 

0.000 

Had 2 dental checkups or more in past 12 months 

45.2 

33.5 

1 1 g *** 

0.000 

Sample size (total = 1,961) 

1,022 

939 



High school students' use of health services (%) 





Uses hospital emergency room as usual source 
of care when sick 

2.5 

3.2 

- 0.8 

0.522 

Has seen dentist for any reason in past 12 months 

93.8 

89.1 

47 ** 

0.021 

Had 2 dental checkups or more in past 12 months 

62.9 

44.1 

18.8 *** 

0.000 

Had health checkup or got shots in past 

12 months 

94.0 

94.3 

-0.3 

0.876 

Sample size (total = 812) 

429 

383 
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Table 4.8 (continued) 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Rewards earned. Years 1-3 ($) 

Total amount earned for health activities b 

2,978 




Sample size 

2,377 





SOURCE: MDRC calculations using data from the Family Rewards 42-month survey and Seedco's 
Family Rewards program data. 


NOTES: Sample sizes may vary because of missing values. 

A two-tailed t-test was applied to differences between outcomes for the program and control 
groups. The p-value indicates the likelihood that the difference between the program and control 
groups arose by chance. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 
5 percent; * = 10 percent. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 
assignment characteristics of sample members. 

Rounding may cause slight discrepancies in calculating sums and differences. 

The items in the health section of the survey were adminstered to a random subsample (N = 
1,961) of the survey respondents. 

The early post-program period refers approximately to the first six months since the time the 
program ended, which is when most respondents to the 42-month survey completed their interviews. 

a The four most commonly reported conditions are listed. 

b Reward amounts are calculated among all program group families, including those who did not 
earn any health rewards. 


Although Family Rewards did not lead to improvements on a range of parents’ health 
outcomes, or on health outcomes that parents reported for their children, one noteworthy sub- 
group finding emerged from an exploratory analysis. 48 Parents who indicated at the time of ran- 
dom assigmnent that they were in “fair” or “poor” health (about 19 percent of the sample) were 
6.2 percentage points more likely than parents in the control group (or almost twice as likely) to 
report that they were in “very good” or “excellent” health at the time of the 42-month survey. 
(See Table 4.9.) They also reported lower rates of having asthma and being treated for it. In 
contrast, those who began the program in better health reported a small increase in the likeli- 
hood of having and being treated for any medical conditions, including asthma, possibly reflect- 
ing previously undiagnosed conditions. Although the effects on asthma are not easily explained 
by other patterns in the data, preexisting health status may be worthy of further exploration in 
future studies of incentives tied to general preventive health care. 


48 This subgroup was not prespecified. 
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Table 4.9 


Impacts on Parents' Receipt of Health Care Services and Health Outcomes, 
by Respondent's Self-Rated Health at the Time of Random Assignment 


Subgroup and Outcome 

Program Control 
Group Group 

Difference 

(Impact) 

P-Value 

Sig. 

Self-rated health is excellent, very good, or good 







at baseline (%) 







Had a health checkup in past 12 months 

89.1 

88.7 

0.3 


0.833 


Has seen dentist for any reason in past 12 months 

86.3 

76.8 

9.5 

*** 

0.000 


Had 2 dental checkups or more 

45.1 

34.9 

10.2 

*** 

0.000 


Excellent or very good self-rated health 

41.7 

40.0 

1.8 


0.463 


Currently has, and is being treated for, 







any medical condition 3 

44.1 

39.8 

4.3 

* 

0.070 

tt 

Asthma 

13.9 

10.8 

3.0 

* 

0.069 

fff 

High blood pressure/hypertension 

21.2 

19.2 

1.9 


0.319 


High cholesterol/high LDL 

6.3 

7.0 

-0.7 


0.584 


Diabetes 

8.2 

7.2 

1.1 


0.433 


Smokes cigarettes 

21.8 

23.6 

-1.8 


0.393 


Obese according to Body Mass Index (BMl) b 

44.1 

44.2 

-0.1 


0.978 


Sample size (total = 1,549) 

797 

752 





Self-rated health is fair or poor at baseline (%) 







Had a health checkup in past 12 months 

93.5 

90.1 

3.4 


0.226 


Has seen dentist for any reason in past 12 months 

81.5 

68.8 

12.7 

*** 

0.004 


Had 2 dental checkups or more 

45.7 

27.8 

17.9 

*** 

0.000 


Excellent or very good self-rated health 

12.6 

6.4 

6.2 

** 

0.039 


Currently has, and is being treated for, 







any medical condition 3 

73.1 

79.1 

-6.0 


0.157 

ft 

Asthma 

19.4 

30.8 

-11.4 

*** 

0.009 

tff 

High blood pressure/hypertension 

42.7 

46.1 

-3.4 


0.491 


High cholesterol/high LDL 

19.7 

18.6 

1.1 


0.790 


Diabetes 

22.7 

23.3 

-0.5 


0.901 


Smokes cigarettes 

26.6 

28.4 

-1.8 


0.697 


Obese according to Body Mass Index (BMl) b 

52.3 

54.2 

-1.9 


0.719 


Sample size (total = 398) 

219 

179 
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Table 4.9 (continued) 

SOURCE: MDRC calculations using data from the Family Rewards 42-month survey. 

NOTES: Sample sizes may vary because of missing values. 

A two-tailed t-test was applied to differences between outcomes for the program and control 
groups. The p-value indicates the likelihood that the difference between the program and control groups 
arose by chance. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; 
* = 10 percent. 

Differences across subgroup impacts were tested for statistical significance. Statistical significance 
levels (Sig.) are indicated as follows: fff = 1 percent; f f = 5 percent; f = 10 percent. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 
assignment characteristics of sample members. 

Rounding may cause slight discrepancies in calculating sums and differences. 

The items in the health section of the survey were adminstered to a random subsample (N = 1,961) 
of the survey respondents. 

a The four most commonly reported conditions are listed. 

b Weight categories are from the National Institutes of Health. Obesity is defined as having a BMI 
of at least 30. About 6 percent of the sample is excluded from this analysis because of missing data. 


Parents’ Employment and Training 

Among parents in the control group, 69 percent had worked in a job covered by unemployment 
insurance (a “Ul-covered” job) at some point during the five-year period for which those data 
are available, but only about 48 percent were working in such a job during any given quarter of 
that follow-up period. Those who worked at all at a Ul-covered job in Year 5 (about 53 percent) 
earned an average of about $24,000 in that year. 

To move out of poverty on a sustained basis without Family Rewards, parents in the 
program would need to do better. The employment and training rewards were intended to en- 
courage and support their work efforts. 

The use of incentives to encourage work has a long history in the United States. The 
Earned Income Tax Credit (EITC) is the best and largest example of a program that increases 
the payoff to work, providing benefits to more than 27 million families. 49 A range of studies 
suggests that expansions in the EITC increased the employment rates of single mothers. 50 A 
number of evaluations have tested programs that provide monthly earnings supplements for 
full-time employment or rewards for sustained work. 51 In general, these programs have in- 
creased work, in some cases by moving more people into the labor market, in other cases by 

49 In the 2013 tax year, the most recent year for which data are available, over 27 million families and indi- 
viduals received the EITC (Center on Budget and Policy Priorities, 2016). 

50 Holt (2006); Eissa and Hoynes (2006). 

5 'Earlier studies are summarized in Michalopoulos (2005). More recent studies include Hendra et al. 
(2011) and Martinson and Hendra (2006). 
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moving people who would have worked anyway to do so more quickly, and in other cases by 
encouraging people who would have worked part time to work full time instead. Another find- 
ing from some studies is that “incentives plus services” are more effective at increasing em- 
ployment than incentives alone, 52 suggesting that some people may need additional help mov- 
ing into work to take advantage of the incentives offer. 

The evidence for training incentives is more mixed. Some programs using incentives 
have increased the rate at which participants engaged in training and received certificates. How- 
ever, the increase in training did not necessarily lead to better labor market outcomes. 53 

The work and training incentives in Family Rewards are distinctive in the United 
States because, unlike work incentives in many other employment and training-focused pro- 
grams, they are part of a larger package of rewards that are intended to motivate positive fam- 
ily practices in education, health, and work. Thus, Family Rewards offers a rare opportunity 
to test the effects of work incentives in the context of other incentives that are not conditioned 
on work. 

To date, none of the major CCT programs in lower- and middle-income countries has 
included rewards for parents’ work or training, although there has been concern that the provi- 
sion of education and health rewards may lead to a reduction in adults’ work effort. In general, 
CCT evaluations have not found negative effects on work, with the exception of one program in 
Nicaragua, where the rewards for education and health were quite large. 54 

Effects on Employment and Training 

• Family Rewards increased the likelihood of self-reported full-time em- 
ployment. It did not increase employment in or earnings from jobs cov- 
ered by the UI system. 

According to the 42 -month survey of parents, the program increased the likelihood of 
working at the time of the interview by 6.4 percentage points above the control group rate of 
49.6 percent. This difference, shown in Table 4.10, was driven largely by an increase in full- 
time work (which the program rewarded). However, the program had no statistically significant 
impact on the average quarterly employment rate in Ul-covered jobs over the five-year follow- 
up period, according to administrative records data. 55 (See Table 4.11.) A very small negative 


52 Michalopoulos (2005); Nunez, Verma, and Yang (2015). 

53 Hendra et al. (2011); Miller, van Dok, Tessler, and Pennington (2012). 

54 Fiszbein and Schady (2009). 

55 Employers report the wages of their workers to the Ul system on a quarterly basis. 
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Table 4.10 


Impacts on Selected Characteristics of Parents' Self-Reported 
Employment in the Final Program Year or Early Post-Program Period 


Outcome (%) 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Currently employed at the time of the survey 

56.0 

49.6 

6 4 *** 

0.000 

Working full time (at least 30 hours per week) a 
Currently employed and job provides 

44.4 

39.5 

4 9 *** 

0.001 

Paid sick days 

33.6 

30.4 

3 2** 

0.026 

Paid vacation days 

35.3 

33.3 

2.0 

0.152 

Sample size (total = 2,966) 

1,543 

1,423 




SOURCE: MDRC calculations using data from the Family Rewards 42-month survey. 

NOTES: Sample sizes may vary because of missing values. 

A two-tailed t-test was applied to differences between outcomes for the program and control 
groups. The p-value indicates the likelihood that the difference between the program and control 
groups arose by chance. Statistical significance levels are indicated as follows: *** = 1 percent; ** 

= 5 percent; * = 10 percent. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre -random 
assignment characteristics of sample members. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Dollar averages include zero values for sample members who were not employed. 

The early post-program period refers approximately to the first six months since the time the 
program ended, which is when most respondents to the 42-month survey completed their 
interviews 

a lf a respondent worked multiple jobs at the time of the interview, then only the characteristics 
of the primary job are reported. (The job at which the respondent worked the most hours is 
considered primary.) 

effect on average quarterly employment (less than 2 percentage points) persisted after the pro- 
gram ended, but the small negative effects on average earnings in Ul-covered jobs during and 
after the program period are not statistically significant. 

Some jobs are not covered by the UI system, such as self-employment (for example, 
providing child care), federal government employment, and domestic work. In addition, the UI 
system also misses informal (casual or irregular) jobs that are never reported to state agencies. It 
is not clear why the effects of Family Rewards would vary across types of employment. Perhaps 
for some parents, non-UI jobs were easier to get, particularly those that offered the full-time 
hours necessary to qualify for the program’s work rewards. Such jobs may also have been more 
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Table 4.11 


Impacts on Employment and Earnings Covered by Unemployment Insurance, 

Years 1 to 5 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Ever employed (%) 

Years 1-5 

66.8 

69.1 

_2 3 ** 

0.021 

Year 1 

56.3 

58.7 

_2 4 *** 

0.008 

Year 5 

51.6 

53.2 

- 1.6 

0.171 

Average quarterly employment (%) 

Years 1-5 

46.8 

47.6 

-0.9 

0.270 

Year 1 

49.1 

50.4 

-1.3 * 

0.098 

Year 5 

44.9 

46.7 

- 1.8 * 

0.082 

Total earnings ($) 

Years 1-5 

62,036 

62,947 

-911 

0.496 

Year 1 

12,154 

12,376 

-221 

0.323 

Year 5 

12,485 

12,746 

-261 

0.476 

Rewards earned. Years 1-3 ($) 

Total amount earned for workforce activities 3 

1,698 

— 

- 

— 

Sample size (total = 4,993) 

2,513 

2,480 




SOURCE: MDRC calculations using data from New York State unemployment insurance (UI) wage 
records and Seedco's Family Rewards program data. 

NOTES: Sample sizes may vary because of missing values. 

A two-tailed t-test was applied to differences between outcomes for the program and control 
groups. The p-value indicates the likelihood that the difference between the program and control groups 
arose by chance. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; 
* = 10 percent. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 
assignment characteristics for sample members. Standard errors were adjusted to account for multiple 
observations per family. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Dollar averages include zero values for sample members who were not employed. 

This table includes only employment and earnings in jobs covered by the New York State UI 
program. It does not include employment outside ofNew York State, nor in jobs not covered by the UI 
system (for example, "off-the-books" jobs and federal government jobs). 

a Reward amounts are calculated among all program group adults, including those who did not earn 
any workforce rewards. 
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attractive options if they were more conveniently located, easier to obtain, or offered more flex- 
ible schedules than Ul-covered jobs. 56 

It is also not clear why the program did not lead to larger increases in all types of em- 
ployment (including Ul-covered jobs), a finding that stands in contrast to the effects of previ- 
ous work incentives programs. Furthermore, subgroup analyses found that the program had a 
statistically significant negative effect on labor market outcomes for parents who entered the 
program with lower education levels and other disadvantages; in other words, they worked 
and earned less than they would have in the absence of the program, according to UI records. 
For example, as shown in Table 4.12, parents who did not have a high school diploma or 
GED certificate when they entered Family Rewards had an average quarterly employment 
rate over the five years of follow-up that was 3 percentage points lower than that of their 
counterparts in the control group, and they earned an average of $2,934 less than those control 
group members over that period (a reduction of almost 8 percent). Similarly, the program led 
to reductions in Ul-covered employment among parents who were not working at the time of 
random assignment and among those whose family incomes were below 50 percent of the 
federal policy level when they entered the program. 

The reductions in Ul-covered work among more disadvantaged parents may be a re- 
sponse to the substantial rewards that these families were earning from the program’s health 
and education domains — an “income effect.” For example, over the entire program period, 
the subgroup with no high school diploma or GED certificate received an average of $7,388 in 
reward payments, with $6,305 (or 85 percent) coming from health and education rewards (not 
shown in Table 4.12). The deep recession and slow recovery that occurred during the fol- 
low-up period would have made it especially difficult for lower-skilled individuals to find 
work, and the income effect may have been amplified for them. Since families were able to 
earn program rewards by completing education or health-related activities, parents who were 
less prepared for the labor market may have relied more on earning what they could from those 
two components of Family Rewards and less on looking for or maintaining full-time work. 57 


56 Health care jobs made lip a large part of the employment that was both reported on the survey and cap- 
tured by the UI data. In contrast, child care jobs made up more than a third of the jobs that were reported on the 
survey but missed by the UI data. This pattern suggests that part of the reason why the survey data indicate that 
Family Rewards increased employment while the UI data do not show that same effect may be because child 
care jobs, which the UI system is less likely to cover, account for a portion of the increase in employment on 
the survey. See Riccio et al. (2013a). 

57 The work disincentive of government transfer programs is one that is predicted by economic theory and 
has been found in practice. Reductions in work effort, although generally modest in size, have been found for a 
range of programs that provide benefits, including SNAP (Hoynes and Shanzenbach, 2012), UI benefits 
(Krueger and Meyer, 2002), and housing subsidies (Jacob and Ludwig, 2012). Jacob and Ludwig, for example, 
found that the use of housing vouchers from a program in Chicago reduced quarterly employment rates by 
about 4 percentage points. 
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Table 4.12 


Impacts on Respondents' Employment and Earnings Covered by 
Unemployment Insurance, by Selected Characteristics at the Time of 


Random Assignment 


Subgroup and Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Sig. 

Education 






High school diploma/GED certificate 
or higher at baseline 






Ever employed, Years 1-5 (%) 

74.1 

76.0 

-2.0 

0.114 


Average quarterly employment. Years 1-5 (%) 

55.1 

54.5 

0.6 

0.561 

tt 

Total earnings, Years 1-5 ($) 

81,411 

81,466 

-55 

0.978 


Sample size (total = 2,863) 

1,404 

1,459 




No high school diploma/GED certificate 
at baseline 






Ever employed, Years 1-5 (%) 

57.5 

60.7 

-3.1 * 

0.081 


Average quarterly employment, Years 1-5 (%) 

35.5 

38.5 

-3.0 ** 

0.018 

tt 

Total earnings, Years 1-5 ($) 

34,579 

37,512 

-2,934 * 

0.074 


Sample size (total = 1,960) 

1,021 

939 




Employment 






Employed at baseline 






Ever employed, Years 1-5 (%) 

87.4 

88.6 

-1.1 

0.236 


Average quarterly employment, Years 1-5 (%) 

71.6 

71.3 

0.3 

0.783 

t 

Total earnings, Years 1-5 ($) 

102,409 

102,931 

-522 

0.809 


Sample size (total = 2,633) 

1,324 

1,309 




Not employed at baseline 






Ever employed, Years 1-5 (%) 

43.5 

47.4 

_3 g ** 

0.040 


Average quarterly employment, Years 1-5 (%) 

18.7 

21.1 

-2.4 ** 

0.036 

t 

Total earnings, Years 1-5 ($) 

16,412 

18,051 

-1,640 

0.268 


Sample size (total = 2,282) 

1,147 

1,135 





(continued) 
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Table 4.12 (continued) 


Program Control Difference 

Subgroup and Outcome Group Group (Impact) P- Value Sig. 


Income 

Income at or above 50% of FPL 
at baseline 

Ever employed, Years 1-5 (%) 

Average quarterly employment, Years 1-5 (%) 
Total earnings, Years 1-5 ($) 

Sample size (total = 3,062) 

Income less than 50% of FPL at baseline 
Ever employed, Years 1-5 (%) 

Average quarterly employment, Years 1-5 (%) 
Total earnings, Years 1-5 ($) 

Sample size (total = 1,931) 


84.0 

84.3 

-0.4 

0.732 

ttt 

64.3 

64.2 

0.1 

0.928 

tt 

91,639 

91,822 

-183 

0.926 


1,584 

1,478 




39.0 

45.3 

-6.3 *** 

0.002 

ttt 

18.3 

21.8 

_2 5 *** 

0.005 

tt 

14,624 

17,514 

-2,890 ** 

0.046 


929 

1,002 





SOURCES: MDRC calculations using data from New York State unemployment insurance (UI) 
wage records. 

NOTES: Sample sizes may vary because of missing values. 

A two-tailed t-test was applied to differences between outcomes for the program and control 
groups. The p-value indicates the likelihood that the differences between the program and control 
groups arose by chance. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 
5 percent; * = 10 percent. 

Differences across subgroup impacts were tested for statistical significance. Statistical 
significance levels (Sig.) are indicated as follows: fff = 1 percent; f f = 5 percent; f = 10 percent. 
GED is General Educational Development. 

FPL is federal poverty level. 


Family Rewards led to small but statistically significant increases in the proportion of 
parents who, at the time of the 42 -month survey, were taking community college courses for 
credit (by 2.3 percentage points), who had earned a trade license or certificate (by 4.2 per- 
centage points), or who held an associate’s degree (by 2.4 percentage points) (not shown in 
tables). 58 Data on the type of license or certificate that was received indicate that a large frac- 
tion of these credentials were for service occupations, including home health aides, nursing, 
and child care. The training rewards are unlikely to have led to such effects on participation 


58 See Riccio et al. (2013a). 
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and credential receipt, because so few individuals received those rewards. Although it is not 
certain, effects on participation and credential receipt may instead have been driven by the 
extra income provided to families through the program, which may have functioned as a type 
of financial aid to either cover tuition or offset necessary child care or transportation costs 
associated with attending classes. 

It seems unlikely that parents who experienced a reduction in earnings were substituting 
education or training for work, hoping that doing so would improve their work opportunities in 
the future. Although this possibility cannot be ruled out, it is noteworthy that for the subgroups 
with reductions in earnings, Family Rewards did not consistently have larger impacts on partic- 
ipation in education or training or on receiving a degree or credential. 
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Chapter 5 

An Overall Assessment 


When launched in 2007, Family Rewards sparked a great deal of interest in and speculation 
about how it would affect participating families. Conditional cash transfer (CCT) programs 
have reduced poverty and increased schooling and health care use in a range of low- and mid- 
dle-income countries. But would a similar idea work in a very different context — a large city 
in a higher-income country? The findings are now in, and they add to the growing body of evi- 
dence not only on the broader concept of CCTs, but also on the promise and challenges more 
generally of using incentives to influence behavior. 

Overall, the program’s effects were more modest than had been hoped. Its largest ef- 
fects were to reduce poverty and material hardship during the three years that the cash rewards 
were offered, particularly for families who entered the program in more severe poverty. But the 
behavioral responses, the key to long-term poverty reduction, were either small or limited to 
subgroups. The program led to some encouraging effects, such as the notable increase in high 
school graduation rates for ninth-graders who demonstrated proficiency in reading before enter- 
ing high school, or the big increase in dental care for all types of families. But it had no effects 
on education among younger students and only a few small effects on other outcomes related to 
health care and parents’ employment. It is worth considering why the original model did not 
produce better results, and whether certain design modifications could make such a program 
more effective. 


Why Didn’t Family Rewards Have Larger and Broader Impacts? 

Although it is impossible to know for sure, it is reasonable to question whether certain design 
choices help explain the program’s mixed results. For example, were the right behaviors and 
achievements rewarded? Were they structured and operationalized appropriately? Were there 
too many rewards overall? Were appropriate dollar values attached to the rewards? Should case 
management have been included? 

The Types of Rewards and How They Were Operationalized 

The designers of Family Rewards had certain expectations about the potential influence 
of each reward within each domain. However, many choices about what activities and accom- 
plishments to reward and how to structure and target the rewards had to be made without the 
benefit of a strong evidence base. The evaluation findings now available are helpful in assessing 
those original choices within each of the three reward domains. 
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Children’s Education 

In the education domain, it is important to distinguish the experiences of younger and 
older students. For elementary and middle school students, Family Rewards linked only two 
types of rewards to student performance: (1) attendance and (2) perfomiance on annual stand- 
ardized math and English language arts (ELA) exams. For the most part, average attendance 
rates generally ranged from 85 percent to 90 percent through eighth grade among control group 
members. 1 As mentioned earlier, recognizing that, at these grade levels, the attendance rewards 
were not making a difference, the program’s designers dropped them for the third and final year 
of program operations. 

There was much more room for improvement when it came to younger students’ per- 
formance on the annual standardized tests: many control group students were not scoring at pro- 
ficiency levels. However, each of these tests occurred only once a year, and the rewards that 
were attached to them may have been too remote to have much motivational power. Young stu- 
dents may not have been able to appreciate fully the connection between working hard during 
the school year, performance on the annual tests, and the rewards that would come from doing 
well on those tests. Furthermore, the test rewards may have also had little salience for young 
students because the payments were made to the parents. Family Rewards, like most other CCT 
programs, relied on parents to tell their children about the incentives. 2 As a result, at least in 
Family Rewards, many of the younger children in the program were not fully aware that incen- 
tives were being offered. In addition, parents may not have understood how to improve their 
children’s study habits or what other concrete steps they could take to help them perfonn better 
in school. 

But it is not clear that this lack of awareness really mattered. The Spark program in 
New York City, which operated at the same time as Family Rewards and offered monetary re- 
wards directly to students in the fourth and seventh grades for better perfomiance on diagnostic 
standardized tests that they took several times during the academic year, also did not produce 
much overall improvement in students’ academic performance. 3 It may be that individually 


'The attendance rates in the Chapter 4 tables do not distinguish excused absences from other absences. 
Thus, some portion of nonattendance is accounted for by excused absences for child illness in both the program 
and control groups. 

2 The decision to rely on parents for communicating with younger children about the program was done for 
practical reasons and because, presumably, many parents would not have wanted their young children to be 
given the sometimes large sums of money that the program provided. 

3 Fryer (2011). Fryer found more promising effects in school-based incentives experiments in which re- 
wards were tied to “inputs” in the education production function (for example, paying second-graders to read 
books and answer quizzes on them, and paying middle-school students to meet various behavioral standards 
along with attendance) than in incentives experiments focused on “outputs,” such as rewarding test scores or 
grades. 
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based monetary incentives simply have less power to motivate younger students, or that other, 
different kinds of behaviors would need to be rewarded. 

High school is a time when attendance rates begin to fall substantially, school perfor- 
mance slips, and dropping out becomes a more common problem than it is in the lower grades, 
especially for lower-income students. By high school, low-income students are nearly three 
times more likely than high-income students to have repeated a grade and nearly six times 
more likely to drop out . 4 Financial incentives should mean more to high school students be- 
cause they have more control over their school-related efforts and their own consumption, and 
it was reasonable to expect that they would have a stronger desire for cash. Family Rewards 
thus paid most of the high school education rewards directly to the students, in whole or in part 
(although many parents still exercised some control over their children’s access to that money). 
It was expected that, even if students were not very interested in school, they may attend more 
regularly, pay more attention in class, complete more of their homework, and study harder in 
order to earn the money — and that their parents would provide more encouragement, support, 
and reminders. 

These rewards did make a difference, but their effects were limited to the subgroup of 
ninth-grade students who were better prepared for high school according to their scores on their 
eighth-grade standardized tests, and especially for those who were proficient in reading. The 
incentives had little effect for those ninth-graders who entered the program less prepared for 
high school. For those students, achieving satisfactory school perfonnance called for a much 
bigger leap than it did for better-prepared students, likely requiring support and assistance that 
went well beyond the offer of financial incentives alone. For example, it may have required 
compensatory instruction or special tutoring; ways of addressing personal or family problems 
that were interfering with school attendance, engagement, and learning; or ways of helping stu- 
dents improve their study habits and focus on schoolwork. 

Behavioral studies find that incentive rewards that are paid closer to the time when the 
desired behavior occurs may be more effective than when payments are delayed, and some 
school research supports that finding . 5 Thus, one weakness of the Family Rewards education 
component may have been that (for administrative reasons) too much time passed between 
achievement of the specified outcome and payment of the reward. For example, attendance re- 
wards were paid two months after they were earned, and payments for achieving the expected 
number of course credits in a year could not be made until after the school year was completed. 


4 Planty et al. (2009). 

5 Levitt, List, Neckermann, and Sadoff (2012) offer evidence that when education incentive payments are 
considerably delayed, they are less effective in improving student performance than those that are received 
with less delay (that is, closer to the behavior or outcome), reasoning that students heavily discount the value of 
the delayed rewards. 
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Similarly, payments for passing Regents exams, for students taking exams in January of a given 
year, would not be made until the summer, when test grades became known. So even though 
families received payments for whatever they earned (across all domains) every two months, 
the rewards for certain school achievements may have been too distant to mean very much to 
some students. Indeed, the evaluation’s qualitative research suggests that some high school stu- 
dents had forgotten about the incentives for certain activities until they received a payment. 

The designers of Family Rewards had originally hoped to assign rewards for report card 
grades, which could have been paid closer to the time when report cards were issued. However, 
concerned that such incentives might put pressure on teachers to grade more leniently, school 
department officials were wary of this approach. They also noted inconsistencies in teachers’ 
standards across different schools. The designers had also considered providing rewards for stu- 
dents to complete homework assignments at a satisfactory level. Given the wide variation in 
how schools and teachers assign homework and track its completion, however, implementing 
and verifying a compliance process was deemed impractical. 

Preventive Health Care Practices 

During the design phase, limited data were available to guide decisions on the health 
component of Family Rewards. Although it was expected that many families in the program 
would be enrolled in Medicaid and the Children’s Health Insurance Program (CHIP), it was 
known that failure to comply with administrative requirements to maintain consistent eligibility 
for public insurance was not uncommon among low-income families, leading to inconsistent 
coverage. It was also expected that many of the parents who might have been working in low- 
wage jobs and may have had the option of purchasing health insurance through their employers 
would not do so because of the cost. It was further expected that many of the families would not 
have a “medical home” and would rely heavily on emergency rooms for nonemergency care. 

These concerns turned out to be largely unfounded — perhaps in large part because of 
the progress that New York City and New York State had made in improving access to health 
insurance and health services for low- and moderate-income populations. As described previ- 
ously, control group rates of preventive health care visits were high, insurance rates were high, 
and reliance on emergency rooms for routine care was low, leaving little room to improve these 
outcomes. (Still, the program had a small positive effect on reducing the likelihood that families 
would have a spell of no insurance coverage.) 

It also turned out that most parents rated their health and their children’s health as good 
to excellent, although many of the parents were contending with various ailments, and about 44 
percent were struggling with obesity. The program may have had a positive effect on self- 
reported health among the subgroup of parents who, when they entered the study, were in fair or 
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poor health (according to an exploratory and less certain analysis). But, overall, encouragement 
for getting annual physicals was not what the majority of families needed. 

The design team (the NYC Center for Economic Opportunity, MDRC, and Seedco) had 
considered tailoring health incentives to participants’ particular health needs, such as treatment 
for asthma, obesity, diabetes, or smoking. It even explored the possibility of offering incentives 
for healthful eating. But the team ultimately rejected such approaches, largely because it could 
see no viable way to monitor and verify families’ compliance with individually tailored incen- 
tives on a large scale, especially in the absence of a case management component to provide 
families with substantial personalized attention. 

Preventive dental care is another story. The consistent positive effects on regular dental 
checkups for adults as well as children show that those types of health-related incentives could 
work in the context of an overall CCT program. It is notable that the control group members 
were less likely to get preventive dental care in the absence of the program. 

Employment and Training 

The failure of Family Rewards to improve employment outcomes for this sample was 
clearly not a matter of “too little room” to make an improvement. By the end of the study, many 
adults in the program group as well as in the control group were not working, and those who 
were employed were largely working in low-wage jobs, many of which were part time. Al- 
though the survey indicated a small positive effect on self-reported employment rates, no gain 
was achieved in Ul-covered jobs. It is possible that the power of an incentive for full-time em- 
ployment, which has worked in other experiments, was undermined by combining it with incen- 
tives that were not conditioned on work (that is, the education and health care rewards). 

Few parents in Family Rewards received the education or training rewards. Although 
about 42 percent took part in some form of education or training during the follow-up period 
(about the same as the control group rate), the fact that the reward could only be earned after an 
approved activity was completed may have been an important limitation. 

It is difficult to conclude that it was wrong to include work incentives in the Family 
Rewards model. Although they did not improve employment and earnings in jobs that were 
covered by unemployment insurance (UI), the positive effects on full-time employment and 
non-UI-covered employment mean that some parents did respond positively to them. Moreover, 
it is possible that the reduction in employment among the more disadvantaged subgroups may 
have been even greater if Family Rewards had included no incentive at all for full-time work. 
Still, the disappointing results clearly suggest that the approach to work in the context of a com- 
prehensive CCT program should be rethought. 
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The Number and Size of Rewards 

Perhaps including 22 different rewards was just too much of a cognitive load — or bur- 
den on their mental resources — for families to manage. For example, the qualitative research 
suggests that some families may have focused most of their attention on rewards that were easi- 
est to achieve, or on activities they were already completing without encouragement from the 
program, rather than striving to earn rewards from other activities on a long list. 

To illustrate this point, consider a program that offers only two incentives, one for fre- 
quent attendance at school and one for perfomiance on standardized tests. Assuming for the 
moment that these two behaviors are the right ones to reward, would a program have more 
widespread effects on education if families had only those two rewards to focus on? Findings 
from at least some of the more targeted incentives programs (discussed in Chapter 4) suggest 
that that might be the case. Conversely, such a pared-down program would not reduce current 
poverty and material hardship nearly as much as Family Rewards did, unless much higher dol- 
lar amounts were attached to those rewards (which could create other problems, as discussed 
below). With more incentives, families had a better chance to earn large cash transfers and the 
program had a better chance of achieving its short-term goal of an immediate reduction in pov- 
erty and hardship — a core objective of a CCT program. In addition, Family Rewards, as a 
comprehensive two-generation model, sought to improve a broad range of human capital out- 
comes, for parents as well as for children. Nonetheless, it seems reasonable, as behavioral re- 
searchers might argue, to pare down the list of 22 rewards that were included in the original 
model so that participants could focus their cognitive resources on a more manageable number, 
and this decision was made for the replication of Family Rewards (discussed in the next sec- 
tion), known as “Family Rewards 2.0.” 

Would attaching larger amounts of money for the main incentives in Family Rewards 
have made a difference? The program designers did not have much evidence to guide them in 
setting the value of the rewards that could be earned. They sought to ensure that the rewards 
could increase participants’ incomes by roughly 25 percent, which was about the amount of 
money that Mexico’s CCT program aimed to add to family incomes. But the designers had to 
balance a variety of considerations, including what was affordable to the demonstration, what 
amounts might be politically acceptable in an ongoing program, and what unintended negative 
consequences could occur if amounts were set too high (for example, for test perfomiance 
among young children, which might have caused parents to put undue pressure on their chil- 
dren). Although extraordinarily high payments for various rewards might have boosted their 
motivational power, practical considerations could not be ignored. 


74 



The Absence of Case Management 

The original Family Rewards design purposely did not include a case management 
component. The goal of the experiment was to determine whether the incentives alone could 
make an important difference, not only in reducing immediate hardship and poverty, but also in 
changing families’ human capital development. Cash transfers were to be provided as long as 
families achieved verifiable compliance with prespecified conditions. In fact, it was possible for 
a family to have very little contact with program staff through the course of the three-year pro- 
gram while being considered “fully engaged” and earning large rewards. 

Qualitative research suggests that Family Rewards may have achieved better results if it 
had provided families with more guidance on how to meet the conditions that earned rewards. 
Many families who were not already meeting the required conditions may not have known what 
steps to take to achieve them, or may have had limited capacity to meet those conditions with- 
out additional help. For example, some parents were at a loss for how to help their children per- 
form better at school beyond giving them general encouragement. Students themselves may not 
have understood how to study effectively or learn material they were struggling to understand. 
Some parents were not quite sure how to secure a full-time job, especially during the recession, 
when jobs were hard to find. The incentives-only approach of Family Rewards had little to offer 
to address those challenges other than general infonnation on the availability of relevant ser- 
vices in the community. 


A Revised CCT Model: Family Rewards 2.0 

With the early evidence from Opportunity NYC-Family Rewards showing some of the 
strengths and weaknesses of the original model, CEO and MDRC sought to try again in 2011, 
even though the final results from the first attempt would not be available for several years. 
They collaborated to design a modified version of Family Rewards and secured funding to test 
it with a new randomized trial. Support came from the federal Social Innovation Fund (SIF) and 
private funds. 6 This time, the program would be tested in two very different locations: the 
Bronx, New York, and Memphis, Tennessee. It operated from 201 1 through 2014. 

The revised model, referred to as “Family Rewards 2.0,” was notable in a number of 
ways. It included many fewer incentives (8 rather than 22), with some dropped after it was de- 
termined that they were not working very well based on emerging findings from the original 
demonstration. The revised program also targeted only families with high school students in 
ninth or tenth grade, since no positive effects on school outcomes had been found for elemen- 
tary and middle school students. It further restricted the selection of families to those who were 
receiving benefits from Temporary Assistance for Needy Families (TANF) or the Supplemental 

6 See Dechausay, Miller, and Quiroz-Becerra (2014). 
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Nutrition Assistance Program (SNAP), as a way to focus attention on families who were among 
the poorest and for whom control group outcomes in the original study tended to be lower than 
for those who were not participating in these benefit programs. This targeting decision was also 
seen as a way of testing a program that had a more direct connection to two of the country’s 
mainstream safety net programs. 

Another important feature of the new model was its “family guidance” component. The 
decision to include this feature was based on the emerging findings from the original study sug- 
gesting that incentives by themselves would not be sufficient to produce big gains in human 
capital development. While participants embraced the goals of the program, many did not know 
what steps to take to meet the conditions that would entitle them to rewards (for example, how 
to perform better in school, or how to get a full-time job). In Family Rewards 2.0, each family 
was assigned a family advisor at their participating Neighborhood Partner Organization (NPO). 
The advisors developed a Family Earnings Plan with every family and sought to meet with 
them at least twice a year to discuss their progress toward earning the rewards. Staff also used 
strategic outreach to engage families who were not earning rewards, and they had access to a 
small “resource fund” (discretionary money) to help families invest up front in services they 
may have needed to help them earn the rewards, such as short-term tutoring, work uniforms, 
transportation to job interviews, or fees for credentialing exams. 

At the same time, Family Rewards 2.0 remained primarily an incentives-driven inter- 
vention. Although the family guidance component would allow staff to counsel families indi- 
vidually on steps they might take to meet the conditions that earned rewards, and to reach out to 
and try to help families who were “low earners” of the rewards, staff were not expected to pro- 
vide in-depth case management. For one thing, they had to contend with relatively high case- 
loads. Each advisor was assigned about 100 families. If each family had only one high school 
student, that raised the number of individuals to 200. Some families also had more than one 
child in high school. Furthermore, the advisors were generalists without any special expertise in 
education, health, or workforce services, although they were prepared to make referrals to other 
services in the community. Thus, it is fair to view Family Rewards 2.0 more as an incentives 
program with some (limited) family guidance than primarily as a case management or service 
program backed up with incentives. 

Family Rewards 2.0 also attempted to make the reward payments more timely (and 
thus more salient) by disbursing payments monthly, rather than every two months. In addi- 
tion, rewarding report card grades, in addition to test scores, provided a more immediate 
incentive for teenagers’ school performance, since these rewards were tied to each report 
card. Students could earn $30 for each A, $20 for each B, and $10 for each C for up to five 
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qualifying courses per report card. Families were eligible for the rewards for three years and 
in some cases a little longer. 7 

Whether this revised approach to a CCT program is more effective than the original 
incentives-only approach remains to be seen. Impact findings for Family Rewards 2.0 will 
be available by the end of 2016. 


Conclusion 

As the first test of a comprehensive CCT program in the United States, Family Rewards at- 
tempted to reduce current poverty and hardship while simultaneously encouraging families to 
make more investments in their human capital to reduce future and next-generation poverty. It 
achieved an important measure of success regarding the first goal and limited results on the sec- 
ond one. In some cases, the incentives led to changes in family members’ efforts and accom- 
plishments, but in most cases they did not. 

It is also important to consider the program’s costs. The Family Rewards cash transfers 
alone cost an average of roughly $3,000 per person per year, while the costs of maintaining the 
payment system and marketing, monitoring, and compliance for each of the reward conditions 
added to that expense. 8 Because it did not produce earnings gains that might have, in turn, re- 
duced reliance on other social transfers (for example, TANF, SNAP, or Medicaid), Family Re- 
wards could not have paid for itself. Moreover, its limited effects on other outcomes that are 
more difficult to monetize — namely, education- and health-related outcomes — do not make a 
compelling case for scaling up the model in its original form. At the same time, because Oppor- 
tunity NYC-Family Rewards succeeded in reducing short-term poverty and material hardship, 
especially for the poorest families, while achieving at least some improvements in human capi- 
tal outcomes, it encouraged further experimentation with a CCT approach in the United States. 
The findings from Family Rewards 2.0 will thus provide a broader empirical foundation for as- 
sessing the merits of such an approach in an American context. 


7 In both New York City (the Bronx) and Memphis, Tennessee, just over 600 families were enrolled in 
each city’s program between September 2011 and February 2012. Twice as many families were recruited for 
the study, with half randomly assigned to a program group, who were eligible for Family Rewards, and half 
randomly assigned to a control group, who were not eligible for the program. With a sample of 1,200 families 
in each city, the evaluation will be able to examine program effects for the combined sample and for each city 
by itself. 

8 See Chapter 3 and Miller and Deitch (2016), available at www.mdrc.org. 
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About MDRC 


MDRC is a nonprofit, nonpartisan social and education policy research organization dedicated 
to learning what works to improve the well-being of low-income people. Through its research 
and the active communication of its findings, MDRC seeks to enhance the effectiveness of so- 
cial and education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 
for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 
Its projects are a mix of demonstrations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 
combination of research and organizational experience to their work, providing expertise on the 
latest in qualitative and quantitative methods and on program design, development, implementa- 
tion, and management. MDRC seeks to leam not just whether a program is effective but also 
how and why the program’s effects occur. In addition, it tries to place each project’s findings in 
the broader context of related research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, and best practices are proac- 
tively shared with a broad audience in the policy and practitioner community as well as with the 
general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy are- 
as and target populations. Once known primarily for evaluations of state welfare-to-work pro- 
grams, today MDRC is also studying public school reforms, employment programs for ex- 
offenders and people with disabilities, and programs to help low-income students succeed in 
college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Children’s Development 

• Improving Public Education 

• Raising Academic Achievement and Persistence in College 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the United 
Kingdom, MDRC conducts its projects in partnership with national, state, and local govern- 
ments, public school systems, community organizations, and numerous private philanthropies. 



